ComCap v4 and V5
The beta file must
The files linked from this page are beta test versions of ComCap. The beta version is made available to those that wish to test new features or have experienced problems in ComCap, to ensure such problems are fixed before an official release. Please make sure you keep your previous version of ComCap, in case you need to downgrade.
ComCap Beta 5.0
This is the last beta is for the next release of ComCap, it is now finished and the help is currently being updated for the final release in a few days.
ComCap5 will have many new major features, those currently planned are:
CopmCap4 was released 15 years ago and has evolved substantially with many new features added without extra cost for existing users. ComCap 5 will have a higher price of £100 (about $130) to reflect the new functionality with an upgrade price for existing ComCap4 licensees. Support will continue for ComCap4 with bug and security fixes and there may be new versions in the future.
ComCap5 licenses are available already, with the license key working in both ComCap 4.15 and later, and ComCap 5.0 for those not prepared to use the beta but wishing to avoid a later upgrade cost.
ComCap4 and ComCap5 may be installed and run side by side on the same PC, but not capturing using the same serial or network ports. Currently, there is no automatic import of ComCap4 settings into ComCap5, this may be added later.
New Features and Fixes - 16th December 2019
1 - ComCap5 will now accept Json and XML capture data formats to save to a database or output as reformatted data, in addition to the existing Fixed Width Columns, Character Separated Columns and Variable Named Columns. Json and XML are set-up on the Capture Settings Data Format tab similarly to Variable Named Columns. After a few lines of original data have been captured, the sample data should show a decoded line with names and values which can be selected in the grid to be added to database columns or output as reformatted data. Note the data format for Json and XML only handles a top level objects and fields from a single record, it can not parse arrays or multiple records, nor nested objects which will be remain as Json (also for XML) objects or arrays. Also, ComCap needs to process the entire block of Json as a single record, so on the Settings Records tab, 'Line or Record End' should be set to Multiple Tags, with the tag for Json generally being '}/n' and for XML '</lasttag>', assuming that the Json record is followed by a newline. These record end settings mean any embedded new line ends within the record are ignored so it is captured as a single line by ComCap.
2 - When they start-up, ComCap5 and ComGen5 now check if the Windows Defender Firewall is running, that firewall rules are set-up and if not add them automatically. Note the tray versions can only add new rules if run with administrator rights (only needed once), the service version is always an administrator. Without firewall rules, ComCap5 and ComGen5 may be unable to use TCP/Server to listen for traffic.
3 - In Channel Settings, General, 'Capture Time Format' specifies how time is handled for each channel, specifically for date and time stamps and log rotation based on time. The options are 'Local Time with daylight time' (default), 'Local Time no daylight time' and 'UTC Time no daylight time'. These options may be used to match time to the format being used in the capture log, and the last two avoid summer time issues. These settings do not apply to the Information Log, which remains system format.
4 - In Channel Settings, Logging, 'Convert Hex to ASCII' specifies that captured data is being sent as hexadecimal text (only 0 to 9 and A to F) and should be converted into ASCII. No checks are made that the text really is hex, ComCap simply removes all spaces and tries to convert whatever arrives to ASCII. This processing takes place before other options so the text can be logged as Raw Data, filtered or any other features.
5 - ComGen5 now reports the SSL/TLS certificate chain on start-up, with a warning if it expires within the next 30 days.
New Features and Fixes - 27th October 2019
1 - Automatic SSL/TLS certificate acquisition and installation from Let's
Encrypt is now supported. Potentially commercial certificates can also be
automatically bought and installed, but this requires account settings to be
added and is not yet available. ComCap is only able to order certificates
for channels available using public domain names on the open internet, not
internal only servers. Again potentially ComCap can issue local certificates
against a private certificate authority, but this also requires more account
settings. Before issuing a certificate, Let's Encrypt will connect to a web
server ComCap runs internally on port 80 of the same IP address used by the
capture or echo channel, so public DNS must point to this IP address and
2 - To configure SSL/TLS certificate ordering, in Common Settings, Network Options tab, select 'Certificate Supplier' as AcmeV2, 'Certificate Product' as 'Let's Encrypt 3 months', 'Certificate Challenge' as 'File - Local Web Server', 'Certificate Private Key Type' as 'RSA 2,048 bits' or 'Elliptic Curve secp256', 'Certificate Sign Digest' as SHA256 and 'Days before Expiry to Order' to 20 or 30 (Let's Encrypt certificates expire after 90 days). 'Certificate Ordering Work Directory' is where the new certificates, private keys and ordering database will be saved. If public internet access requires a proxy server, the 'Proxy URL' should be entered as http:/server:port. These settings are common to certificate ordering for all ComCap channels, where domain specific information is specified.
3 - SSL/TLS certificates are only used by TCP Server capture channels, or TCP Server Echo to Remote. TCP Client and UDP do not need certificates. In Channel Setting, the Network Options tab has certificate settings for TCP Server and Multi TCP Server and HTTP Server, if SSL is enabled (in Common Settings). If a SSL Server Certificate file name is specified and exists, details will be shown half way down the tab, including the common name and expiry date. Public Certificates may be ordered from Let's Encrypt by clicking the 'Order Public Certificate Now' button, provided the correct Common Settings are configured. If the 'SSL Server Certificate' field is empty, the default certificates directory will be used. 'Certificate Domain Name' must be completed, which is the public domain pointing to the IP address configured for capture. The IP address may be a public IP range, or a local IP NAT forwarded from a public router. Before the certificate order process starts, ComCap checks the domain name is publicly available. If 'Private Key Password' is left blank, it will be automatically set to 'password' since P12 certificates need a password. Once these details are completed, clicking the button will start the order progress with details appearing in the ComCap Information Log, it usually takes about 30 seconds to complete a Let's Encrypt order with no more interaction needed. Once settings are saved, the new SSL certificate will be loaded automatically. There is another button 'Create Local SSL Certificate' which will create a self signed certificate for the 'Certificate Domain Name' which is installed automatically, but there is no check this domain is publicly accessible. Finally, the check box 'SSL Certificate Automatic Public Ordering' allows Multi TCP Server channels only to re-order certificates without any manual intervention 30 days (or less) before they expire, provided the ComCap is capturing data. On the Echo tab, 'TCP Server Echo to Remote' has identical certificate and ordering settings to capture settings.
4 - Improved the start-up of 'TCP Server Echo to Remote' logging any errors with SSL certificates. Start-up of 'Multi TCP Server' now uses the correct configured SSL Certificate Authority Bundle file.
5 - In the main capture window and logs, any domain name specified for TCP Server are now shown as well as the IP address.
New Features and Fixes - 25th September 2019 - 5.0
1 - In Channel Settings, the Filters tab is now called Capture Filters, and includes a new option 'Required Phrases (case sensitive), any one'. This allows a list of phrases to be specified, at least one of which must be found in the record for it to be saved. This might be used for remote authentication so only records with a specific mobile IMEI or IP address are accepted, or perhaps including a month. Beware the phrase is not column specific and may be found as part of something unwanted, so the longer the better. Alert phrases have moved to a new tab Capture Alerts.
2 - In Channel Settings, a new Capture Alerts tab replaces the previous
simple list of phrases to which the same alert be sent, instead allowing a
different SMS number, mail address, subject and body to be specified for
each phrase found. If Body is left blank, the whole line is sent as
previously. To send to multiple SMS or emails, set-up duplicate phrases. All
alerts still appear
3 - In Channel Settings, the General tab has a new 'Reformat Data' tick box that enables the 'Data Format' tab (similarly to Save to Database) and a new 'Reformatted Data Format' selection. These options now allow ComCap to save captured data in a different format, for instance fixed width lines of data may be saved as comma separated quoted columns for easier processing. Reformat options are Tab Separated, Comma Separated Quoted Variable and Variable Named, with Json and XML to be added soon. On the Data Format Tab, once some data has been captured in original format, Click the 'Sample Log Up' button to display a line of data, if it's CSV one format row will be created for each column found named Column 1, etc. For fixed length columns, a single format row is created and the 'Add New Row' button should be clicked to add extra rows with the required data position and data length. The Data Names for the columns only matter if the new format includes column names. Beware of editing the Data Format using the Reformatted Output, the sample data will be wrong.
4 - In Common Settings, SMS tab, added a second HTTP bureau The SMS Works at https://thesmsworks.co.uk/ for sending SMS. cheaper than Kapow at less than 2p each and allows the sender ID to be freely set as either a mobile number or text defaulting to 'ComCap Alert'. Once you have opened an account, generate an API Key and Secret which is a five lines of Json text that should be copied to the 'The SMS Works Json Logion Lines' field instead of the login used by Kapow. Once an SMS has been sent, the number of account credits remaining will be shown in the Info Log.
5 - If Windows Explorer crashes and the taskbar disappears and is recreated, the ComCap icon is no longer lost.
6 - ComCap5 no longer attempts to start twice on start-up.
7 - In Channel Settings, Records, the default minimum line length is now 2,000 characters, to avoid long lines being broken.
8 - In Common Settings, Capture Logging tab, 'Send Alert on Stop' has been changed to 'Send Alert on start and Stop' so ComCap sends an alert on start-up as well.
9 - Fixed a silly bug that caused the capture log window to clear if the line contained the letters FF. Undisplayable characters are now shown as space rather than being removed.
10 - Beware the new configuration settings for Reverse scrolling, Automatic certificate ordering and Windows firewall are not yet implemented, next beta.
New Features - 22nd May 2019 - 5.0
1 - Support has been added for TLSv1.3, the first improvement in SSL for about 10 years, using OpenSSL 1.1.1b.
2 - In Common Settings, Network Options, 'SSL Client Security' now specifies the SSL security level for all TCP/IP Client (including email) to ensure that minimum security standards are enforced. The options are:
If there are problems accessing servers using earlier beta versions of TLSv1.3, set the Security Level to TLSv1.2 Only. Updated SSL root certificates with more of those commonly used.
3 - In Common Settings, Network Options, 'SSL Server Security Level' now specifies the SSL security level for all TCP/IP Servers to ensure that minimum security standards are enforced. The options are:
Ideally use TLSv1.1 minimum since TLSv1 has been deprecated by the PCI DSS council. It's no longer necessary to specify DHParams, SSL Server ECDH Key or SSL Allowed Host Names. There is a 'Certificate Domain Name' which should set to the common name of the SSL server certificate specified, to allow it to be validated correctly.
4 - In Channel Settings, Network Options, the same new 'SSL Client Security' and 'SSL Server Security Level' options may be set for that TCP/IP Server SSL capture on that channel only, and separately on the 'Echo' tab for Echo to TCP Client and Server.
5 - ComGen5 includes the same 'SSL Client Security' and 'SSL Server Security Level' options as ComCap4 on the Network Options tab where the SSL server certificate is specified.
6 - In 5.0 Channel Settings, General, when using Data Format 'Character
Separated Columns (CSV)', it is now possible to set the 'Column Quoting
Character' as well as the 'Column Separating or Delimiting Character'.
Generally, the separator is a comma (,) and the optional quoting character a
double quote (") which should appear after the separator if the column
contains the separating character. If the quoting character appears in the
column it needs to be escaped by doubling, ie "". But if the quoting
character regularly appears in a column, then it should be changed to
something else that does not appear.
New Features and Bug Fixes 13th June 2018 - 5.0
1 - Initial support for capture using the HTTP protocol has been implemented, for GET and PUT requests where the data is supplied URL encoded. POST requests will be added shortly, as will more validation over page names. For HTTP capture, in Common Settings, Network tab, create a 'TCP Multi Server channel and set Service to HTTP.
2 - Support has been added for TLSv1.3, the first improvement in SSL for about 10 years which is expected to become an RFC standard in the next few weeks. Currently ComCap uses a beta version of OpenSSL 1.1.1 supporting draft 28, which is becoming widely accepted. Medium SSL security level now sets TLSv1.1 minimum since TLSv1 has been deprecated by the PCI DSS council. Updated root certificates with more of those commonly used.
New Features and Bug Fixes 25th October 2017 - 5.0
1 - Added a new sessions window that allows easy viewing of remote TCP connections to TCP Multi Server channels, rather than checking back through the logs. The main capture window right click menu has a new option 'Sessions Window' which will open a free floating resizable window listing remote sessions since capture was started. Currently sessions are identified only by remote IP address, so will not distinguish multiple connections from the same remote device, but this will be improved for the next beta. For each session, the remote IP address and port are shown, them the total lines captured, last line time, when the session started and ended (if over), session id, and amount of data captured, all the things shown in the main capture window for individual capture channels. Active remote sessions are coloured light green, closed sessions light red. There is a right click menu that allows control of individual remote sessions similarly to the main capture window, specifically Close Remote Session, Resend Start Command, Send Data, View Map Window and Log GPS Info, the last two for mapping channels only. The sessions window is automatically refresh when a remote connections opens or closes, and also at an optional frequency for progress updates, between every 5 and 300 seconds.
2 - The right click menu in the main capture window has a new option 'Close Remote Session' for TCP channels that, after a prompt, allow closing of the remote connection, perhaps if it seems to have stalled, forcing the remote device or ComCap if TCP Client to attempt a reconnection.
3 - Fixed a long term nasty logging bug when there was a problem writing to the information log, that caused ComCap to exhaust memory and crash. This would happen if Main and Alternate logging directories were the same, so the second log files could not be opened and the usual solution of using a different file name would fail since the name is the same in both directories. The original bug is now fixed, and checks made to ensure Main and Alternate logging directories are different. This issue did not effect error handling Capture Logging.
4 - Added support for two more GPS vehicle and personal trackers data formats: Xenun TK102/103 and WondeX/TK5000. The Xenun TK102/103 format is essentially the NMEA RMC sentence, preceded by date/time and mobile number, followed by useful stuff from other NMEA sentences like satellite count, mobile IMEI and cell station stuff. The WondeX/TK5000 format used by VT-10, VT300 and other devices is a simple format with IMEI, time, co-ordinates, speed and direction. Both these data formats are TCP/IP server only. They are also used by a Android application MyLiveTracker by Michael Skerwiderski available free from Google Store: https://play.google.com/store/apps/details?id=de.msk.mylivetracker.client.android.
5 - Fixed a capture log display issue when changing tab that caused the top line of the log to be sometimes lost.
6 - ComCap and ComGen now use the latest OpenSSL 1.1.0f version with the latest security fixes.
New Features and Bug Fixes 17th March 2017 - 5.0
1 - When archiving (and perhaps zipping) capture logs to a new directory on rotation, any customised capture sub-directory specified is now used as well. A customised capture sub-directory typically uses a partial date, so a new sub-directory is created monthly or daily, to avoid large numbers of files in the same directory. It is specified as part of the 'Custom Log Name Mask', ie 'yyyy-mm"\capture-"yyyymmdd".txt"' will cause a daily sub-directory to be created.
2 - The main window File menu has a new option 'View Mail Queue' that opens a new window showing any items pending in the Mail Queue, and allowing them to be cancelled if necessary. In general, email is sent within a few seconds assuming there is an internet connection, but sometimes email can get stuck in the queue and never sent.
3 - Fixed a bug that meant email being sent that failed all retries was not deleted from mail queue, but caused logging to go wild with errors.
4 - When entering SSL certificate file names in Preferences and Channel Settings, it's now possible to specify a password if the private key is encrypted. When entering SSL server certificate, private key and optional intermediate chain are now validated to ensure the SSL server will start correctly, for instance the private key matches the certificate, the certificate is issued by an intermediate and/or by a root CA authority, and that certificates have not expired. These checks also take place when the SSL server is started and may prevent capture being started. Previously, only PEM certificates were supported, ComCap now also supports DER, PFX, P12, P7B, CER and CRT file formats. PFX/P12 (PKC12) is convenient because it can contain certificate, private key and intermediates so only a single file needs to be used.
5 - Fixed a validation issue introduced in ComCap 4.13 that prevented an IP address being specified to sent the Information Log remotely using UDP or TCP Client.
Bug Fixes 5th December 2016 - 5.0 and 4.17
1 - Fixed two problems with Email channels, they no longer always listen on port 587 and listening now correctly ceases when a channel is stopped.
2 - ComCap4 and ComGen now use the new OpenSSL 1.1.0c version with the latest security fixes. The OpenSSL DLLs are now digitally signed and checked when being loaded, for improved security. Checking of SSL certificates has been improved.
3 - If ComCap4 and ComCap5 are installed together, they will now both start automatically when Windows boots, if so configured.
4 - If an unexpected error happens during capture, usually with TCP Multi Server, an alert is now sent and capture restarted.
New Features 20th September 2016 - 5.0
1 - Reworked the way the capture and information log windows are refreshed so that much higher update rates are possible. Previously, only a few hundred lines per second could be displayed, now this is a few thousand per second. Viewing data at such rates is not really practical since the scrolling windows only show the last 5,000 lines (configurable), but this change means ComCap will no longer become non-responsive under heavy traffic. No data was ever lost, this merely relates to viewing live data.
2 - Tested the new 'TCP Multi Server' channel type with 2,000 simultaneous SSL sessions, each sending one line per second. Although a large number of connections are supported, opening each new SSL connection does take a finite duration limiting the number of new connections per second. Testing seemed to show the SSL connection limit to be about 100 per second on a decent PC, but this may vary significantly depending on hardware. Non-SSL connections have lower overhead, so many more per second. Most TCP clients will retry a refused connection, so should get connected when traffic is slower.
3 - ComGen5 has been rewritten to support more functionality creating test data streams to exercise ComCap5. There is a new ComGen5 Settings window with the same four tab that were previously in the main window. The Network tab has a new protocol 'Multi TCP Client' which will generate the number of clients as specified in the new 'Clients' column, up to 2,000 at present. Each client session will terminate after 'Lines/Session' have been sent. To avoid starting all these sessions at once (which most servers will be unable to handle), the General tab has a new setting 'Maximum New TCP Client Sessions Per Second' to restrict that number, typically 100 per second, or less for slower servers. The tab also has 'Random Gap Between Sessions (secs)' with 'Minimum' and 'Maximum' being specified, defaulting to 5 and 60 seconds, so that repeat sessions are staggered. Any sessions that fail to connect first time will retry after 'Wait Before Retry (secs)' for 'TCP Client Retry Attempts'.
4 - ComGen5 will now create HTTP requests to test web servers. There are three new Data Type Formats, HTTP GET URL, HTTP POST URL, HTTTP PUR URL which should be used with TCP Client or Multi TCP Client. The Remote Host and Port are set in the Network grid, with the page and parameters specified as the Data Type Text, ie: HTTP GET URL and '/index.html' will get that URL, multiple URLs can be accessed by separating them with |, ie: '/page1.html|/page2.html|/page3.html'. For both POST and PUT, parameters follow the URL and ?, ie: /upload.html?Text test line from ComGen Id \? on \p at \t session \S serial \#6 with the escape dynamically processed identically to Simple Text format. ComGen5 does not support pipelining but waits for a response to each request, which is reported in the result column of the main progress grid. In Setting, General, the 'HTTP User Agent' for the requests may be specified. The request and first response will always be logged, but generally subsequent requests are not logged, see below, just counted.
5 - ComGen5 now include optional log files. The Settings, General tab has a tick box 'Write Log File' and 'Directory' which if completed cause daily log files to be created. 'Log All Connection Attempts' defaults to unticked to reduce the amount of logging with repeated connections and disconnections, so only the first and last are generally logged, but may be ticked for full logging of all attempts.
6 - ComCap5 and ComGen5 now use the new OpenSSL 1.1.0 version with the latest security fixes and features, including ChaCha20-Poly1305 encryption cipher suites, X25519 ecliptic curve for ECDH ciphers and OCB and CCM mode ciphers
7 - Improved error messages when setting up Network channels so that trying to use a target host name instead of an IP address says this only works if Family is set to 'Any'.
New Features 21st July 2016 - 5.0
1 - In ComCap4, one TCP Server channel needs to be created to support each simultaneous remote client, usually with a few spare in case clients do not cleanly disconnect and block a channel. While adequate for a few remote clients, this is very tedious for hundreds of channels. To simplify all this, a new channel type 'TCP Multi Server' has been added which can accept a large number of simultaneous remote clients, all capturing data to the same log file and optionally a database.
2 - Capturing from multiple remote clients to a single channel raises issues of how to identify data from each client, and which remote clients are connected. ComCap can currently add a sequential serial number for each captured record, and a new sequential 'Session Id' has been added, which is incremented for each new remote capture session, when ComCap accepts a new incoming TCP connection or makes an outgoing TCP connection. This Session Id is shown in the Information Log for all activities for all TCP Multi Server remote connections, as follows (cut down a little):
Session 31 [1 of 2] From Address 192.168.1.80:52375, Started at
23-Jul-2016 14:26:36, etc
The above information is logged hourly is so specified in Common Settings, Log Files or when the Status button is clicked.
The Session Id is also supported in Channel Settings, Logging, 'Add Custom Text to Captured Lines' using the escape \S, with a specified number of digits. It can also be added to a database by ticking 'Update Session_Id Column'. The other escape that will identify the remote client is \R for Remote IP Address.
3 - The next ComCap5 beta will add a new window that shows all the current TCP Multi Server remote connections with details data from each and the last line captured, and will allow control over each remote session, such as disconnecting it.
4 - TCP Multi Server channels support Data Loss checking, but only cause disconnection of the remote client rather than restarting the channel. The normal 'Idle TCP Server Close Session Timeout' may also be used, but is only based on a simple timeout, rather than Data Loss which can have different timeouts at night if there is less data.
5 - Most ComCap features work with TCP Multi Server, except 'Remove Printer Control Sequences'. Some have limitations, currently there is no GPS Map display since it would jump around too much tracking multiple vehicles, but this may be improved in a later beta. If Echo is used to forward data to another computer, any received data is ignored. 'Commands to Send upon Start and Stop Capture' are supported for each remote client, but not right click 'Send Data' to send specific data to a remote. Merging and Filtering from TCP Multi Server are not supported.
6 - When a capture log is rotated (updating completed), it may now be zipped to save space. In Channel Settings, Files, 'Archive Capture Log on Rotation' must be ticked and an archive directory specified, then also tick 'Zip Archive Logs' and optionally specify a password with which to encrypt the zip to prevent unauthorised access. Currently this zipping process temporarily blocks ComCap5 displaying more data, but this should be for less than a second unless the file is very large. If this delay becomes a problem, rotate capture logs more often to make them smaller. If the delay becomes a serious problem, zipping will be done using a background thread so capture continues.
7 - Various minor SSL improvements have been made. SSL certificates can now be specified separately for TCP Server capture and TCP Server echo, since these may be using different host names. Likewise, validation of certificates for TCP Client capture and TCP Client Echo can now be specified separately. A new 'SSL Security Level' has been added for TCP Server capture and TCP Server echo, with three levels: High is TLS/v1.2 only, Medium is TLS/v1 or better, Low is SSL/v3 or TLS/v1 or better. ComCap no longer supports SSL/v2.
8 - In Channel Settings, Data Loss, Check for Data Loss, if 'Send Alert' is ticked, it's now possible to enter a free format 'Alert Message' that will be emailed or sent as SMS, instead of the standard ComCap generated message, which may be more meaningful and useful. Currently the email address and subject are fixed, but these will become configurable in a later beta.
Differences from Comap4 - upgrading guide
A few settings have moved around to make it easier to configure ComCap5.
1 - SSL for network capture has moved from Channel Settings, Network to Common Settings, Network as a SSL tick box in the network grid.
2 - The Channel Settings, Network tab has been split into Channel Settings, Common and Channel Settings, Echo. Separate SSL server certificates may now be specified for TCP Server capture and TCP Server echo. 'Idle TCP Server Close Session Timeout' has moved from the General to Common tab.
3 - GPS configuration has moved from Channel Settings, General, to Common Settings, Network, Service select GPS, and Common Settings, Serial Ports, GPS tick box.
4 - ComCap5 settings are in new directories and files to allow ComCap4 to
co-exist. Configuration settings are now in:
5 - ComGen5 settings needed to be specified new, there is no upgrade from ComGen4, sorry.
If you notice any problems with this beta release, please href="../mailer.asp?comcap">email with details of the problem and the date of the beta version you are using.
Return to ComCap Home Page
Copyright © 2019 Magenta Systems Ltd, England. All Rights Reserved.