Partial Retrieve from Encrypted Store

In a previous post we covered the retrieval of data from a store. It was described that the data at rest and in transit were not encrypted. The reason was that the store was deployed in facilities behind the firewall. Access from the outside was not allowed.

As time changed, data was only encrypted in transit using secure sockets or HTTPS. Given that facilities had to allow remote access, it made sense to encrypt the data at rest. When a client application would request data, the storage server would decrypt the file and send it to the requesting client via a secure protocol.

The issue illustrated on the previous post was due to the fact that the data requested with an offset would start being decrypted from the specified offset; not the beginning of the file. Of course such approach was reasonable when the data at rest was not encrypted. The software has been updated and it starts decrypting at the start of the file as it should. No data is transferred until the requested offset is reached. At that point the data is transferred as requested.

Following is the original file:

C:\> type c:\temp\list.txt
14feb5f0be5a26e887b05703fefa0000
14feb5f0be5a26e887b05703fefb0001
14feb5f0be5a26e887b05703fefb0002
14feb5f0be5a26e887b05703fefc0003
14feb5f0be5a26e887b05703fefc0004
14feb5f0be5a26e887b05703fefd0005
14feb5f0be5a26e887b05703fefe0006
14feb5f0be5a26e887b05703fefe0007
14feb5f0be5a26e887b05703ff000008
14feb5f0be5a26e887b05703ff000009
14feb5f0be5a26e887b05703ff01000a
14feb5f0be5a26e887b05703ff01000b
14feb5f0be5a26e887b05703ff01000c
14feb5f0be5a26e887b05703ff02000d
14feb5f0be5a26e887b05703ff02000e
14feb5f0be5a26e887b05703ff02000f
14feb5f0be5a26e887b05703ff030010
14feb5f0be5a26e887b05703ff030011
14feb5f0be5a26e887b05703ff040012
14feb5f0be5a26e887b05703ff040013
14feb5f0be5a26e887b05703ff050014
14feb5f0be5a26e887b05703ff050015
14feb5f0be5a26e887b05703ff050016
14feb5f0be5a26e887b05703ff060017
14feb5f0be5a26e887b05703ff0b0018
14feb5f0be5a26e887b05703ff0c0019
14feb5f0be5a26e887b05703ff0c001a
14feb5f0be5a26e887b05703ff0d001b
14feb5f0be5a26e887b05703ff0d001c
14feb5f0be5a26e887b05703ff11001d
14feb5f0be5a26e887b05703ff12001e
14feb5f0be5a26e887b05703ff12001f
14feb5f0be5a26e887b05703ff120020
14feb5f0be5a26e887b05703ff120021
14feb5f0be5a26e887b05703ff120022
14feb5f0be5a26e887b05703ff130023
14feb5f0be5a26e887b05703ff130024
14feb5f0be5a26e887b05703ff140025
14feb5f0be5a26e887b05703ff140026
14feb5f0be5a26e887b05703ff140027
14feb5f0be5a26e887b05703ff140028
14feb5f0be5a26e887b05703ff140029
14feb5f0be5a26e887b05703ff15002a
14feb5f0be5a26e887b05703ff15002b
14feb5f0be5a26e887b05703ff15002c
14feb5f0be5a26e887b05703ff15002d
14feb5f0be5a26e887b05703ff15002e
14feb5f0be5a26e887b05703ff16002f
14feb5f0be5a26e887b05703ff160030
14feb5f0be5a26e887b05703ff160031
14feb5f0be5a26e887b05703ff160032
14feb5f0be5a26e887b05703ff160033
14feb5f0be5a26e887b05703ff170034
14feb5f0be5a26e887b05703ff170035
14feb5f0be5a26e887b05703ff180036
14feb5f0be5a26e887b05703ff180037
14feb5f0be5a26e887b05703ff180038
14feb5f0be5a26e887b05703ff180039
14feb5f0be5a26e887b05703ff18003a
14feb5f0be5a26e887b05703ff19003b
14feb5f0be5a26e887b05703ff19003c
14feb5f0be5a26e887b05703ff19003d
14feb5f0be5a26e887b05703ff19003e
14feb5f0be5a26e887b05703ff1a003f
14feb5f0be5a26e887b05703ff1a0040
14feb5f0be5a26e887b05703ff1a0041
14feb5f0be5a26e887b05703ff1b0042
14feb5f0be5a26e887b05703ff1c0043
14feb5f0be5a26e887b05703ff1c0044
14feb5f0be5a26e887b05703ff1c0045
14feb5f0be5a26e887b05703ff1d0046
14feb5f0be5a26e887b05703ff1d0047
14feb5f0be5a26e887b05703ff1e0048
14feb5f0be5a26e887b05703ff1e0049
14feb5f0be5a26e887b05703ff1e004a
14feb5f0be5a26e887b05703ff1f004b
14feb5f0be5a26e887b05703ff20004c
14feb5f0be5a26e887b05703ff20004d
14feb5f0be5a26e887b05703ff21004e
14feb5f0be5a26e887b05703ff22004f
14feb5f0be5a26e887b05703ff230050
14feb5f0be5a26e887b05703ff230051
14feb5f0be5a26e887b05703ff240052
14feb5f0be5a26e887b05703ff250053
14feb5f0be5a26e887b05703ff260054
14feb5f0be5a26e887b05703ff270055
14feb5f0be5a26e887b05703ff280056
14feb5f0be5a26e887b05703ff2b0057
14feb5f0be5a26e887b05703ff2b0058
14feb5f0be5a26e887b05703ff2c0059
14feb5f0be5a26e887b05703ff2c005a
14feb5f0be5a26e887b05703ff2d005b
14feb5f0be5a26e887b05703ff2d005c
14feb5f0be5a26e887b05703ff2d005d
14feb5f0be5a26e887b05703ff2d005e
14feb5f0be5a26e887b05703ff2e005f
14feb5f0be5a26e887b05703ff2e0060

The complete file is stored and encrypted. This is accomplished by the server.

C:\> casstore c:\temp\list.txt
casstore <<< STORING files using CASOpen(), CASWrite() and CASClose() line: 625
14feb5f0be5a26e887b05c1a82bd97bb - 0.26 seconds

As before, we retrieve a few lines from the file. Note that we requested from the start (no offset). This was and continues to work as illustrated:

C:\>casretrieve 14feb5f0be5a26e887b05c1a82bd97bb c:\temp\14feb5f0be5a26e887b05c1a82bd97bb.txt -o -do 0 -dl 102
casretrieve <<< RETRIEVING bifile using CASRetrieve() line: 616
casretrieve <<< c:\temp\14feb5f0be5a26e887b05c1a82bd97bb.txt -  0.87 seconds

We now display the contents of the retrieve file to verify we have not introduced a new issue.

C:\>type c:\temp\14feb5f0be5a26e887b05c1a82bd97bb.txt
14feb5f0be5a26e887b05703fefa0000
14feb5f0be5a26e887b05703fefb0001
14feb5f0be5a26e887b05703fefb0002

Now let’s request three lines but skipping the first one.

C:\>casretrieve 14feb5f0be5a26e887b05c1a82bd97bb c:\temp\14feb5f0be5a26e887b05c1a82bd97bb.txt -o -do 34 -dl 102
casretrieve <<< RETRIEVING bifile using CASRetrieve() line: 616
casretrieve <<< c:\temp\14feb5f0be5a26e887b05c1a82bd97bb.txt -  0.24 seconds

This was not working before due to the issue with encryption at rest. Let’s take a look:

C:\> type c:\temp\14feb5f0be5a26e887b05c1a82bd97bb.txt
14feb5f0be5a26e887b05703fefb0001
14feb5f0be5a26e887b05703fefb0002
14feb5f0be5a26e887b05703fefc0003

As we can verify, the second, third and fourth lines were properly retrieved.

When making changes to software, no matter how simple and innocuous they might seem, it is imperative to run tests to verify that the changes do not introduce new issues / bugs. By properly architecting, designing, implementing and testing the software, customers tend to appreciate the quality of the product / service.

If you have comments or questions regarding this or any other post, please leave me a note. I will respond as soon as possible.

Regards;

John

Follow me on Twitter:  @john_canessa

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.