The use of string tests might not be a common process when testing software; perhaps it should.
So what is String Test?
String Tests are used to test software features when already integrated in the final software and before it is sent to production. When features in the software are being developed, engineers write unit tests to make sure that the different functions and classes work the way they were intended. These tests are typically white box and have reduced scope.
As the software grows and there are multiple dependencies, when changes are made to different parts, it is customary to run redundancy tests at a high level to make sure all is well. String Tests define procedures to check specific feature at a high level. If a test fails then the issue / bug needs to be addressed or the string test needs to be updated.
String Tests should be easy to understand and conduct by the development and QA teams. In general a test written by developers for developers tends to make many assumptions which may or may not be understandable / correct to QA and end users. This is one of the reasons after all tests pass by the development team, then the QA should run the same and additional ones making sure knowledge of the software internals is not needed to run and determine if the tests pass or fail.
The idea when writing a string test is to make it easy for the individual running the test to execute them and verify results. String tests should take a few (5 to 15) minutes on average to execute. The tester should not need a deep understanding of the software system to conduct and most important verify them.
Natural language is not precise. Because of this fact, when writing a string test the steps and results should be simple to understand, perform and verify. It is said that if you cannot explain something; you do not know it well enough. The development team should be able to write string tests that are simple to understand and replicate by others and the results should always be easy to verify. If they are not, then the software probably has an issue. It is not a good idea to allow testers to interpret the meaning and results of a test. Such practice is a total waste of time and resources.
So how do we write a good string test? Like anything in software, it is a science and an art. Let’s take a stab at writing a good string test. Please note that with time things change. The development team may decide to add (not delete) sections. If that is the case, make sure they are always included, the sections are precise, the process is simple to understand and conduct and the results are easy to verify.
To make sure string test files are easily accessible by testers, write them using a text editor (not a word processor). By doing this the string test files may be copied and displayed on any platform without the need of additional software.
String tests should always have at least the following sections:
| Objective |
| Setup |
| Stimulus |
| Results |
Each section should be as short as possible but must exactly convey what is expected to be done.
Label each test case to they are easily located within the string test file.
The Objective sections details what the test attempts to prove (or disprove).
The Setup section describes what needs to be done in preparation for the actual test.
The Stimulus section contains what needs to be done with the software to test a single feature. This section may comprise of a single or multiple steps.
The Results section should be simple to describe so the person conducting the test would be confident that the results described match the results obtained. No interpretation should be left to the tester.
Let’s go through writing a string test case for a storage server that I have a 
very good understanding of. The storage server in question accepts requests to store, query, and retrieve digital objects (e.g., text files, images, videos, etc). In this case we would like to write a string test to verify the operation of “sending a merged bitfile to a storage server”.
A bitfile is an object in the storage server associated with a digital file of any type (i.e., movie, image, word processing document, medical image, etc) in a domain space. For example, when a patient gets an x-ray using one of dozens of different types of equipment (e.g., MRI, CT, etc) the resulting DICOM files representing the images may be sent to a storage server for safe keeping.
The following screen capture shows the contents of this string test case extracted from a text file (e.g., test_casxfer.st):
************ Test Case 1 ************ Objective: Verify that a merged bitfile may be sent and properly stored in the storage server. A merged bitfile is created by taking a set of files (i.e., text, video, images, etc) and merging them locally (not at the storage server) into a merged bitfile. A client is needed to issue the CLI request and a storage server running (in this case) on a Windows machine. Setup: On one or two computers have the client (CLIs) and storage server software properly installed (these items are available in any standard test computer). Make sure you use the cashelp.exe CLI to check on other CLIs (e.g., casping.exe, casversion.exe, casquery.exe, etc) that may be of use to help determine and possibly address issues when running this test case. 1. From a command prompt start the storage server: $ sdm -debug 2. Verify you are using the proper version of the storage software. $ casversion ... If the expected software version does NOT match; stop and install the proper software version. 3. Using a set of 5 to 10 files of any type (files of different types may be mixed together if desired; but is not required) create a local merged bitfile. $ casmergefiles ... Stimulus: Send the locally created merged bitfile to the storage server. 1. Transfer the merged bitfile to the storage server. $ casxfer ... Results: Verify that the merged bitfile is stored in the storage server. 1. Check that the merged bitfile is in the storage server. $ casmergeinfo ... 2. Retrieve the merged bitfile from the storage server and extract its contents. $ casextractallmerged ... Compare the contents of the merged bitfile with the retrieved files. 3. List the extracted files (the contents of the merged bitfile were extracted to the specified folder): C:\>dir "c:\temp\Retrieve Folder" 03/17/2018 10:24 AM 15 14feb5f0be5a26e887b05aad2778347a.txt 03/17/2018 10:24 AM 16 14feb5f0be5a26e887b05aad2778347b.txt 03/17/2018 10:24 AM 17 14feb5f0be5a26e887b05aad2778347c.txt 03/17/2018 10:24 AM 31 14feb5f0be5a26e887b05aad2778347d.txt 03/17/2018 10:24 AM 32 14feb5f0be5a26e887b05aad2778347e.txt 03/17/2018 10:24 AM 33 14feb5f0be5a26e887b05aad2778347f.txt 03/17/2018 10:24 AM 192 14feb5f0be5a26e887b05aad27783480.txt 03/17/2018 10:24 AM 960 14feb5f0be5a26e887b05aad27783481.txt 03/17/2018 10:24 AM 960 14feb5f0be5a26e887b05aad27783482.txt 03/17/2018 10:24 AM 960 14feb5f0be5a26e887b05aad27783483.txt 03/17/2018 10:24 AM 3,298 14feb5f0be5a26e887b05aad27783484.txt 03/17/2018 10:24 AM 12 14feb5f0be5a26e887b05aad27783485.txt 03/17/2018 10:24 AM 3,298 14feb5f0be5a26e887b05aad27783486.txt 03/17/2018 10:24 AM 3,298 14feb5f0be5a26e887b05aad27783487.txt 03/17/2018 10:24 AM 1,674 14feb5f0be5a26e887b05aad27783488.txt 03/17/2018 10:24 AM 192 14feb5f0be5a26e887b05aad27783489.txt 4. List the files that were included when the merged bitfile was generated (in this example the following list of files was used): C:\>type c:\temp\list_of_text_files.txt c:\temp\text files\15_chars.txt c:\temp\text files\16_chars.txt c:\temp\text files\17_chars.txt c:\temp\text files\31_chars.txt c:\temp\text files\32_chars.txt c:\temp\text files\33_chars.txt c:\temp\text files\ach.txt c:\temp\text files\ach_1.txt c:\temp\text files\ach_2.txt c:\temp\text files\ach_3.txt c:\temp\text files\guid_list.txt c:\temp\text files\hello_world.txt c:\temp\text files\list.txt c:\temp\text files\my_guid_list.txt c:\temp\text files\query_times.txt c:\temp\text files\retrieved.txt 5. Verify that the list of files matches the available files (in this case the following folder was used): C:\>dir "c:\temp\text files" 06/24/2016 09:21 AM 15 15_chars.txt 06/24/2016 09:22 AM 16 16_chars.txt 06/24/2016 09:22 AM 17 17_chars.txt 06/24/2016 09:23 AM 31 31_chars.txt 06/24/2016 09:23 AM 32 32_chars.txt 06/24/2016 09:24 AM 33 33_chars.txt 02/09/2016 04:03 PM 192 ach.txt 02/09/2016 02:54 PM 960 ach_1.txt 02/09/2016 02:58 PM 960 ach_2.txt 02/09/2016 03:38 PM 960 ach_3.txt 04/12/2016 09:16 AM 3,298 guid_list.txt 06/24/2016 08:15 AM 12 hello_world.txt 04/07/2016 05:02 PM 3,298 list.txt 05/04/2016 09:45 AM 3,298 my_guid_list.txt 04/05/2016 01:17 PM 1,674 query_times.txt 02/29/2016 06:57 PM 192 retrieved.txt 6. As an added bonus one may select a random number of the files and compare their contents. They should be identical down to the byte count and contents.
Along the way, it was mentioned that to get help with the storage server CLIs one may use cashelp.exe to get the specifics on their use. Following is a screen dump of the cashelp.exe CLI:
C:\>cashelp
casappendtrailer APPEND trailers to bitfiles in a folder.
cascache Display the STATE of all configured disk caches.
cascachestostore Remote disk cache(s) that may be used to store bitfile(s) for specified group ID.
caschangegroup Issue a request to a iCAS server to CHANGE the group of the specified GUID.
cascheckbitfiles CHECK a bitfile or set of bitfiles specified by GUID in a specified iCAS server.
cascheckcache Check the contents of a disk cache.
cascheckcifs Check the contents of the CIFS_TBL table against the specified file system.
cascheckdigest Check if the specified digest matches the file and a GUID in a iCAS server.
caschecklic Check if the specified license is valid.
cascopymissing Copy damaged \ missing bitfiles to iCAS from off-line disk cache.
cascountbitfiles Returns the count of bitfiles in all the folders of all disk caches.
casdelete Delete a GUID or set of GUIDs from the specified iCAS server.
casdumpmerged Dump the contents of a LOCAL merged bitfile.
casexec Execute a command on the host of a iCAS server.
casexercise Store, query, retrieve and delete a test file to / from a iCAS server.
casextractallmerged Extract from a MERGED bitfile the files to the specified path.
casfindissues Check if bitfiles exist in configured disk caches and if they are damaged.
casgendigest Compute the MD5 digest for the specified file.
casgetevents Register to receive iCAS server notification events.
casgetguids Return all GUIDs for a specified date and group (also see caslistguids).
casguidsfordate Return all GUIDs for a specified date.
casguidstopaths Return the full paths to specified bitfiles in disk caches.
cashelp List all the CLI commands with a brief description.
caslistguids Return all GUIDs for a specified date and group (also see casgetguids).
caslistmissing Check all the bitfiles for the specified date generating a list of missing ones.
casloadbalance Return load balancing information on a iCAS server cluster.
caslogoff Log OFF a user to a specified iCAS server.
caslogon Log ON a user from a specified iCAS server.
casmerge MERGE a set of bitfiles into a single bitfile.
casmergefiles MERGE a set of files into a MERGED bitfile without a iCAS server.
casmergeinfo Provide information about a MERGED bitfile.
casmigrate Remigrate all bitfiles from a iCAS server based on migration configuration.
casmvbfs Move all bitfiles from a source to a destination disk cache.
casmsmq Test the operation of iCAS server events using the MSSQ.
casparsesyngoblob Extract from a Siemens Syngo blob all DICOM objects.
casping Verify network connectivity to a specified iCAS server.
casprefetch Prefetch a bitfile from a a remote iCAS server.
caspull Pull missing bitfiles from remote into local iCAS (needs configured PULL group).
caspurge Purge from the disk cache(s) temporary or migrated bitfiles.
caspush Push into an iCAS a file/bitfile skipping a head and/or a tail.
casquery Issue a query for the specified GUID to the specified iCAS server.
casreadtrailer Read the trailer from the specified bitfile.
casremovetrailer Remove bitfile trailers from specified bitfiles.
casretrieve Issue a retrieve request for the specified GUID to the specified iCAS server.
casretrievebyoffset Retrieve a single bitfile from a MERGED bitfile.
casroute Route a set of one (1) or more bitfiles from one to another iCAS server.
casshutdown Issue a shutdown request to the specified iCAS server.
cassnmp Request information via SNMP about system services running on a computer.
cassplit SPLIT a merged bitfile into its bitfiles.
casstore STORE to the specified iCAS server the specified file.
cassynch SYNCHRONIZE the contents of the specified iCAS servers.
cassyngoimages EXTRACT a set of DICOM objects from a bitfile stored by a Siemens Syngo PACS.
castestio TEST I/O operations on disk caches.
castime TIME transfer operations to an iCAS.
castouch TOUCH the specified set of files.
casversion Retrieve software VERSION information from the specified iCAS server.
casxfer TRANSFER a single bitfile from a client to an iCAS server.
To obtain ADDITIONAL information for each command line interface (CLI); from a command prompt
invoke the command using the "-h" help flag:
e.g., casxfer -h
How do we determine if the storage server is up and running? Using the cashelp.exe CLI we can easily determine that we could use casping.exe:
C:\>casping SockToSendReceive <<< connect WSAECONNREFUSED remoteIP ==>192.168.1.110<== localIP ==>192.168.1.110<== port: 4444 line: 2404 file ==>c:\sencorsource\source\sock.c<== SockSendCommand <<< SockToSendReceive WAR_CONNECTION_REFUSED serverIP ==>192.168.1.110<== serverPort: 4444 requestType: 0x00000001 line: 3715 file ==>c:\sencorsource\source\sock.c<== CASPing <<< SockSendCommand WAR_CONNECTION_REFUSED line: 16536 file ==>c:\sdm\source\sdmapi.c<== casping <<< CASPing WAR_CONNECTION_REFUSED serverIP ==>192.168.1.110<== serverPort: 4444 line: 524 casping <<< retVal: -1002 line: 629 file ==>c:\sdm\source\casping.c<==
It seems that the storage server is not running so we probably need to start it. Before we do we could see is there is something else we could get out of the casping.exe CLI by issuing:
C:\>casping -h
casping [iCAS TCP/IP]
This command issues a "ping" to the specified iCAS.
To determine connectivity to a storage server one should first try the TCP/IP ping utility.
This may or may not work if the computer hosting the storage server has "ping" disabled.
To determine connectivity at the storage server (application) level one should use the casping CLI.
-a <ACCESS code>
-d <DELAY in seconds>
-h display this HELP screen
-i send an ICMP packet to the computer hosting the storage server
-p <storage server PORT>
-s <storage server TCP/IP>
-t TESTING ONLY - casping CLI hungs
-u <USER name>
-v run this command in VERBOSE mode
This command issues a “ping” to the specified iCAS.
To determine connectivity to a storage server one should first try the TCP/IP ping utility.
This may or may not work if the computer hosting the storage server has “ping” disabled.
To determine connectivity at the storage server (application) level one should use the casping CLI.
Now let’s determine how to start the storage server:
C:\>sdm -h SDM -install to install the service SDM -remove to remove the service SDM -debug <-recover> to run as a console app for debugging StartServiceCtrlDispatcher being called. This may take several seconds. Please wait.
Looks good so let’s do it:
C:\>sdm -debug CmdDebugService <<< Debugging SDM !!! SDM <<< starting line: 314 ... SDM <<< _WIN32 defined line: 1079 SDM <<< _WIN64 NOT defined line: 1087 :::: :::: :::: SDM <<< enableCheckGroupTable: 0 (bool) controlled by EnableSDMCheckGroupTbl line: 5506 SDM <<< pollForRemoteGUIDs: 0 (bool) controlled by PollForRemoteGUIDs line: 5623 SDM <<< casXferEncrypted: 0 (bool) controlled by CASXferEncrypted line: 5742 SDM <<< !!!! The SDM service; part of the iCAS; is READY to process requests !!!! line: 5756 SDM <<< LISTENING for requests line: 5879 ... :::: :::: ::::
Please note that for this test we wish to start the storage server (sdm.exe) interactively. The storage server can also be started as a service but all messages would be written to a log file. It is easier to run it interactively for debugging purposes.
After the storage server is started we could use the casping.exe CLI:
C:\>casping casping <<< iCAS reply from 192.168.1.110
Or:
C:\>casping -i casping <<< ICMP reply from 192.168.1.110 casping <<< iCAS reply from 192.168.1.110
Seems like we get a reply. All is well.
When testers are about to create a merged bitfile; help is always at their fingertips:
C:\>casmergefiles -h
casmergefiles file_1 ... file_n -m <full_path_for_merged_bitfile>
This command LOCALLY (no need to access an iCAS server instance) merges into
a merged bitfile a set of one (1) or more files (NOT bitfiles) specified by
full path name.
This CLI returns the GUID associated with the newly created merged bitfile.
It is up to the caller to transfer the generated merged bitfile to an instance
of an iCAS server using the casxfer.exe CLI.
MERGE_MIN_GUID_COUNT: 1 MERGE_MAX_GUID_COUNT: 32768
file_i A file to be merged
-d DELETE the original bitfiles that were merged
-f full path to a text FILE holding a list of bitfiles to merge
(you may use two '-' characters to comment out entries)
-g <merged file GROUP ID>
-h display this HELP screen
-l full path to a text file in which the LIST of GUIDs and merged GUID
(last entry) are returned
-m <merged bitfile PATH>
-r <REPOSITORY full path>
-v run the command in VERBOSE mode
The same holds true when atester is ready to use the casxfer.exe CLI:
C:\>casxfer -h
casxfer bitfile
Transfer (NOT store) a single or multiple bitfile(s) (NOT file(s)) from a
client to an iCAS instance.
This CLI is typically used to transfer from a client a newly created
merged bitfile (using the CLI casmergefiles.exe) to an iCAS server.
bitfile FULL PATH to a single bitfile to be transfered to an iCAS
-a <ACCESS code>
-c <path to a disk CACHE folder>
-d <DELAY in seconds>
-f <FILE holding a list of full paths of bitfiles to transfer>
-h display this HELP screen
-p <iCAS PORT>
-r REMOVE bitfile in iCAS before sending (ignores returned status)
-s <iCAS TCP/IP>
-u <USER name>
-v run the command in VERBOSE mode
And also when checking the results of the test:
C:\>casmergeinfo -h
casmergeinfo merged_GUID
Issue a query for the specified merged bitfile (by GUID)
to the specified storage server.
merged_GUID GUID of a merged bitfile
-a <ACCESS code>
-g <GUID of interest in the merged bitfile>
-f <FILE in which to write the list of GUIDs>
-h display this HELP screen
-p <storage server PORT>
-s <storage server TCP/IP>
-u <USER name>
-v run the command in VERBOSE
As you can see, a person with almost no experience in the architecture, design or implementation of the storage server and its API is able to generate off-line a merged bitfile, send it to an instance of the storage server and then verify that the contents of the stored merged bitfile match 
what were specified.
Always keep in mind that natural language is not exact and precise. Repeating the same thing over and over never helps. You need to express the ideas using different words. If you cannot describe something; you do not understand it. Simple tools (e.g., text editor) can be successfully used in software development. You do not need expensive tools to produce high quality software. Always keep in mind the KISS (Keep it Simple and Short) rule.
If you have comments or questions regarding this or any other post in this blog, please leave me a comment. I will respond to it as soon as possible.
Happy software development;
John
Follow me on Twitter: @john_canessa
Good day JOHN. (hope that this post finds you well, considering the Global pandemic amongst other things)
I read the article on “String Testing” , Interesting and somewhat really nice Technical explanation. I would just like to bring to your attention, I noticed a small TYPO in the article in the following paragraph.
Natural language is not precise. Because of this fact, when writing a sting test the steps and
Kind regards
Nizam.
Good day Nizam. Doing well. Thanks for asking. Hopefully you are doing well too.
I appreciate your comment.
Did a search on the article and found not one, but two instances of the same typo (sting -> string).
I cannot afford a reviewer for my posts. That said; I should pay more attention, and I will.
Keep safe and have fun reading, experimenting and learning.
Regards;
John