ComGen Data Stream Generator

ComGen5 is an application delivered free with ComCap5, designed for testing ComCap. It generate various types of test streams using any or all of the PC serial COM ports and hundres of network streams, UDP Client, TCP Client or TCP Server, using SSL/TLS if needed, with both IPv4 and IPv6 protocols.

ComGen is a low overhead application, and will run for hundreds of hours, generating millions of lines of test data, to thoroughly test ComCap. It may be installed on the same PC as ComCap, or copied onto remote PCs and run there.

The main ComGen window shows data generation channels in the left pane, and log on the right. There are buttons to Start and Stop data generation, to hide the window and to access the Settings window.

When all the settings are completed, click Start to commence data streaming. The data generation window shows one row for each stream of data being generated, a tick box indicates if that channel is currently streaming data, the total number of remote clients connected to that channel, the total number of lines generated, and the actual text of the last line (which may help getting the Data Type escapes correct).

ComGen Settings has five tabs: Data Types, Network Channels, Serial Port Channels, General and Network Options. Once these settings have been specified, OK or Apply should be clicked to save the settings. The Settings window is only available when not generating data.

Data Types defines the actual lines of data that will be streamed, which may be the common between two or more channels 15 data types are supplied with ComGen, and others may be created as needed to test ComCap.

Simple Text	Simple text including escape sequences as listed below.
Masked Text	Masked Text including escape sequences and also time and data mask characters formatted in the same way as ComCap capture Files. Note all text is treated as a mask unless surrounded by quotes.
Text File - Once	Text data from a file, sent one line at a time when CRLF is found. Sent once, then stops.
Text File - Repeat	Similar, but repeats on reaching the end.
Binary File - Once	Binary data from a file, sent 100 bytes at a time ignoring CRLF. Sent once, then stops. Sending 10 lines per second will send 1Kbyte of data per second.
Binary File - Repeat	Similar, but repeats on reaching the end.
HTTP GET URL	Create HTTP GET requests, see below.
HTTP POST URL	Create HTTP POST requests, see below.
HTTP PUT URL	Create HTTP PUT requests, see below

\#6	Serial Number, where the digit is the number of digits to use, with leading zeros.
\S	Session number when a channel sends to more than one session at a time.
\d	Date and time, ie 21-Jun-2006 20:10:12. Note this is a fixed date format, if more flexible formatting is needed use the Time Stamp Each Line option above.
\t	Time, ie 20:10:12
\o	ISO date and time, ie 2006-06-21T20:12:11, recommended for database capture
\s	Space, used as a separator at the start or end of the escaped text, not necessary within the text
\p	PC Name (NETBIOS),ie MYCOMPUTER
\i	Local IP address, ie 192.168.44.55
\m	Channel Description, note spaces may cause problems
\n	New line (CRLF), not generally recommended
\f	Form Feed (FF)), not generally recommended
\c	Carriage Return (CR)), not generally recommended
\l	Line Feed (LF), not generally recommended)
\\	Backslash (\)
\e	Escape (ESC)), not generally recommended
\xnn	Any hex code where nn is 01 to FF
\?	ComGen channel ID
\h	Syslog header

When using Text or Binary File formats, the button at the end of the row may be clicked to select a file to stream. The file may be actual data captured by ComCap such as CDRs (two small sample files are supplied) or any other data, optionally including the escape sequences above.

No CRLF ticked means no line ending is sent for Simple Text or Masked Text, which is usually how UDP data is sent.

There are three new Data Type Formats, HTTP GET URL, HTTP POST URL, HTTTP PUR URL which should be used with TCP Client or Multi TCP Client. The Remote Host and Port are set in the Network grid, with the page and parameters specified as the Data Type Text, ie: HTTP GET URL and '/index.html' will get that URL, multiple URLs can be accessed by separating them with |, ie: '/page1.html|/page2.html|/page3.html'. For both POST and PUT, parameters follow the URL and ?, ie:

with the escape dynamically processed identically to Simple Text format. ComGen5 does not support pipelining but waits for a response to each request, which is reported in the result column of the main progress grid. In Setting, General, the 'HTTP User Agent' for the requests may be specified. The request and first response will always be logged, but generally subsequent requests are not logged, just counted.

Network configuration is very similar to Common Settings, Network Channels, Local and Remote IP and Port at the same. To simplify this grid, retry attempts have been made common to all channels and are specified on the Network Options tab, see below. Note that SSL is set-up in the grid, unlike in ComCap4. A remote host name can only be specified if Family

UDP Client	Sends UDP packets to the specified IP address and port.
TCP Server	Listens for connections from remote TCP Clients and sends lines of data back to those clients, note this is effectively Multi TCP Server and up to five client can connect at a time, all of them being sent identical data. May use SSL, but the same certificate applies to all channels.
TCP Client	Sends lines of data to a remote TCP Server, one client connection only. May use SSL.
Multi TCP Client	Sends lines of data to a remote TCP Server, up to 2,000 client connections as specified in the Client column. May use SSL.

The channel description is optionally included in the Data Types line using the escape \m as part of the line sent remotely.

For Multi TCP Client, how client connections should be created, up to 2,000, The Network Options tab specifies how many new clients are created each second, defaulting to 100. Note there is overhead in Windows setting up new connections, particularly with SSL, so you need to restrict how many are generated to avoid them being rejected.

Specifying a non-zero number for TCP Client and Multi TCP Client causes the TCP session to be disconnected after that many lines of data have been sent, to simulate remote data capture devices that continually connect, send some data and then disconnect.

Clicking Data Type will drop down a list of all the Data Types specified on the previous tab, from which one should be selected.

How Often is used to specify the frequency with which data is generated for this channel, selected from the drop down menu as Each Second, Each Minute, Each Hour and Random (between one and 10 seconds) with the Often By multiplier. So 5 by Each Second is five lines of text each second, 30 by Each Minute is one line every two seconds. ComGen has been tested with up to 50 lines per second, and is limited to 60 lines per second. Note that all lines are sent at the start of the second, rather than being spread out evenly.

This column specifies the Next Serial Number that will be used for the data, if it includes the \# mask.

Specifying non-blank text causes this text to send at the start of the session before any other data is sent, to simulate remote data capture devices that identify themselves in some way (like the Ecov). If this data should be sent as a separate line, use a /n escape to send CRLF, ie <TEST>\n

Serial RS232 port configuration is very similar to that in Common Settings, Serial Ports. Note that ComGen only support 8 bit’s for data with 1 stop bit and no parity. Some virtual ports may have strange names like CNCA2, but these will work identically to those starting with COM.

The channel description is optionally included in the Data Types line using the escape \m as part of the line sent remotely.

Clicking Data Type will drop down a list of all the Data Types specified on the previous tab, from which one should be selected.

This column specifies the Next Serial Number that will be used for the data, if it includes the \# mask.

The 'Auto Start' tick box specifies that streaming starts as soon as ComGen is run. If ComGen is to run when Windows starts, a shortcut should be placed in the Windows Startup folder.

Ticking this box enables IPv6 support for ComCap, allowing IPv6 addresses to be specified in various settings screens.

As detailed on Network configuration, TCP Client normally sends a ping to a remote server, which is echoed back if the server exists. Some firewalls and routers may be configured to block pings, causing ComCap to fail to receive the echo and be unable to connect. This tick box bypasses the ping, allowing an immediate connection attempt to the remote server. The penalty is Windows takes about 40 seconds to time out a failed connection attempt, compare to 10 seconds for ping.

When sending HTTP requests, the User Agent sent with the requests, defaults to 'Mozilla/5.0 (Windows NT; ComGen5)'.

This tick box specifies that serial data should only be sent if the remote computer has raised Data Terminal Ready.

A tick box that specifies ComGen should keep a disk log file of all activity, as well as displaying it on the screen,

If ticked, increases the amount of logging by including repeated connections and disconnections, normally only the first and last are logged.

The same SSL certificate will be used for all TCP/IP Server streams. There is no remote server certificate checking for TCP/IP client.

For TCP Client only, specifies the number of connection attempts that should be made to the remote computer before failing. Zero attempts means never stop, but keep retrying for ever, other the maximum attempts is 99.

For TCP Client only, specifies the gap between a failed connection and the next retry attempt, with a minimum of 10 seconds and maximum of 300 seconds (five minutes). Note a connection attempt takes a minimum of 10 seconds, but about 40 seconds if ping is disabled. The more frequent the connection attempts, the more potential network traffic that is carried, but the lesser probability of lost data.

For TCP Client only, this option enables automatic keep alive messages to be transmitted every few seconds, defaulting to 20 seconds. Keep alive is only needed when there are long gaps during data capture, and a router or firewall may disconnect the TCP/IP connection due to inactivity (perhaps after 5 or 10 minutes). This option should not be needed on LANs. Setting seconds to zero disables Keep Alive, which may upset some routers.

For Multi TCP Client only, to avoid starting all multiple sessions at once (which most servers will be unable to handle), specifies the number of new sessions per second, typically 100, or less for slower servers.

For Multi TCP Client only, two fields to specify the gap between new sessions as a range of seconds as Minimum and Maximum, defaulting to 5 and 60 seconds. This causes repeat sessions to be staggered. Any sessions that fail to connect first time will retry after 'Wait Before Retry (secs)' for 'TCP Client Retry Attempts'.

Specifies the SSL security level to ensure that minimum SSL/TLS security standards are enforced. The options are:

None	All protocols and ciphers, any key lengths
SSLv3 Only	SSLv3 only, all ciphers, any key lengths, MD5 hash
TLSv1 Only	TLSv1 only, all ciphers, RSA/DH private keys => 2,048 bits
TLSv1.1 Only	TLSv1.1 only, all ciphers, RSA/DH private keys => 2,048 bits
TLSv1.2 Only	TLSv1.2 only, all ciphers, RSA/DH private keys => 2,048 bits - recommended
TLSv1.3 Only	TLSv1.3 only, all ciphers, RSA/DH private keys => 2,048 bits
TLSv1 or Better	TLSv1 or later, all ciphers, RSA/DH private keys => 1,024 bits
TLSv1.1 or Better	TLSv1.1 or later, all ciphers, RSA/DH private keys => 1,024 bits
TLSv1.2 or Better	TLSv1.2 or later, all ciphers, RSA/DH private keys => 2,048 bits - recommended
Backward Ciphers	TLSv1 or later, backward ciphers, RSA/DH private keys => 1,024 bits, ECC keys => 160 bits, no MD5, no SHA1 hash
Intermediate Ciphers	TLSv1.1 or later, intermediate ciphers, RSA private keys => 2,048 bits, ECC keys => 224 bits, no RC4 ciphers, no SHA1 hash
High Ciphers, 2048 keys	TLSv1.2 or later, high ciphers, RSA private keys => 2,048 bits, ECC keys => 224 bits, no RC4 ciphers, no SHA1 hash - recommended
High Ciphers, 3072 keys	TLSv1.2 or later, high ciphers, RSA private keys => 3,072 bits, ECC keys => 256 bits, Forward Security forced
High Ciphers, 7680 keys	TLSv1.2 or later, high ciphers, RSA private keys => 7,680 bits, ECC keys => 384 bits, Forward Security forced

The default security level is 'TLSv1.2 or Better' which is the PCI DSS council standard and recommended by major browsers. Generally the only reason to support old protocols or low security standards is to access 10 year or older servers that only supported those old protocols. Likewise, all SSL certificates have used 2,048 bit minimum private keys for several years and any older ones should have long expired (except some root certificates). The SHA1 hash was used to sign old certificates now replaced by SHA2 (aka SHA-256). Some SSL ciphers are potentially open to attack, but may still be needed to access very old servers that don't support anything better. Private keys with RSA 3,072 bits are the minimum recommended by NIST for use after year 2030, larger RSA keys increase the size of SSL certificates and thus the handshaking for each SSL connection.

Note if the security level is set too high, an SSL/TLS connection may just fail without any sensible explanation

Specifies the actual file name of the PEM Bundle File, the file supplied with ComCap with about 289 CA root certificates is:

Extra PEM root certificates can be added to this file if needed, manually, or it can be replaced with a file containing only certificates that should be trusted, perhaps self signed certificates. Note the CA file is also used to validate SSL/TLS server certificates.

These settings specify the SSL/TLS certificate for all TCP Server channels, without which they will not start, see SSL/TLS and Certificates. The certificate may be shared with ComCap5 channels or other applications.

Specifies the SSL/TLS server X509 certificate file, which may contain one or more certificates in various formats and a private key. Sometimes separate files are used for server certificate, private key and optional intermediate certificates, but using a bundle keeps them together for simplicity. The two bundle formats supported are PEM (which contains base64 ASCII) and PFX or P12 which is PKC12 binary format. Certificate only files may be PEM, DER, or P7 format. Sometimes PEM files have a CER extension.

Note ComGen checks hourly for any new certificate files being available and will automatically load them without needing to restart the channel, provided the file names are unchanged.

If the SSL Server Certificate was not a bundle including a private key, allows a SSL Server Private Key X509 PEM file to be specified, see SSL/TLS and Certificates which must match the Servr Certificate. If the private key is encrypted, the password should be specified here, this also applies to bundles.

Defaults to the PC host name which may include a domain, but needs to be the Domain Name assigned to the IP address of the TCP Server, for which the SSL/TLS server certificate has been issued. For internal systems with internally issued certificates, the Domain Name may simply be the computer host name.

If the SSL Server Certificate was not a bundle including intermediates, allows a default SSL Certificate Intermediate X509 PEM file to be specified, see SSL/TLS and Certificates. Most server certificates are signed by the supplier using an intermediate certificate, which is in turn signed by a trusted root CA certificate, so this intermediate needs to be supplied to allow the chain to be verified against a trusted root.

Specifies the SSL security level to ensure that minimum SSL/TLS security standards are enforced. The options are:

None	All protocols and ciphers, any key lengths
SSLv3 Only	SSL3 only, all ciphers, any key lengths, MD5 hash
Backward Ciphers, TLS1 or Later	TLSv1 or later, backward ciphers, RSA/DH private keys => 1,024 bits, ECC keys => 160 bits, no MD5, no SHA1 hash
Intermediate Ciphers, TLS1.1 or Later	TLSv1.1 or later, intermediate ciphers, RSA private keys => 2,048 bits, ECC keys => 224 bits, no RC4 ciphers, no SHA1 hash
Intermediate Ciphers FS, TLS1.1 or Later	TLSv1.1 or later, intermediate ciphers, RSA private keys => 2,048 bits, ECC keys => 224 bits, no RC4 ciphers, no SHA1 hash, Forward Security forced
High 112 bit Ciphers, TLS1.2 or Later	TLSv1.2 or later, high ciphers, RSA private keys => 2,048 bits, ECC keys => 224 bits, no RC4 ciphers, no SHA1 hash - default.
High 128 bit Ciphers, TLS1.2 or Later	TLSv1.2 or later, high ciphers, RSA private keys => 3,072 bits, ECC keys => 256 bits, Forward Security forced
High 192 bit Ciphers, TLS1.2 or Later	TLSv1.2 or later, high ciphers, RSA private keys => 7,680 bits, ECC keys => 384 bits, Forward Security forced
TLSv1.2 or Earlier	TLSv1.2 or earlier, intermediate ciphers, RSA private keys => 2,048 bits, ECC keys => 224 bits, no RC4 ciphers, no SHA1 hash, Forward Security forced
TLSv1.3 Only	TLSv1.3 only, intermediate ciphers, RSA private keys => 2,048 bits, ECC keys => 224 bits, no RC4 ciphers, no SHA1 hash, Forward Security forced

While using the highest level of security is always best, this may prevent older clients connecting to ComCap. If clients attempt to connect with the latest TLSv1.3 protocol but fail, try setting security to 'TLSv1.2 or Earlier', the latest is not always the best. Note that the server SSL certificate must have a key length of the minimum the security level requires, or capture will not start. At the time of writing, the recommended default is 'High 112 bit Ciphers, TLS1.2 or Later', but this may change to 128 bit in a few years.