The Nanoscale World

HSDC Format

rated by 0 users
Answered (Verified) This post has 1 verified answer | 6 Replies | 4 Followers

Top 75 Contributor
14 Posts
Points 158
Bruker Employee
DigestMigrator posted on Wed, Jun 2 2010 2:39 PM

Normal 0 false false false

Hi All,

 

I am trying to figure out a way to batch export High Speed Data Capture (hsdc) files generated with Veeco’s Nanoscope software. Unfortunately the Nanoscope software does not support batch export for those files.  Does anybody know a software that can open and batch export hsdc files?

 

Thanks for your suggestions,

Cosmin

 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Cosmin Mihai

Biomedical Engineering Department

The Ohio State University

Columbus, OH, USA.

 

email: cosmin.mihai@osumc.edu

Answered (Verified) Verified Answer

Top 10 Contributor
280 Posts
Points 6,221
Bruker Employee

Hi Cosmin,

There are separate sections in the header for each data channel.  Each of the sections starts with a line "\*Ciao HSDC list".  You will find different Data offsets, lengths and scales for each channel.

I looked around and found an Igor procedure to read HSDC files!  

Here is a description of the procedure:

 

Igor function to read a Nanoscope High Speed Data Capture (HSDC) file and load the data into Igor waves with appropriate scaling.  The native Nanoscope file is read and the ASCII header is parsed to get scaling factors while the binary part is read directly, so no export is required.

See the comments at the beginning of the file for more details on usage.

This function was created by Ben Ohler in June 2006.  The code has been tested with Igor v5 and Nanoscope v7 and v8 HSDC files.

Please rename the extension from .txt to .ipf prior to using it

 OpenHSDC.txt

I think that will do what you need.  If you make improvements, please post them back here.

--Bede

All Replies

Top 10 Contributor
280 Posts
Points 6,221
Bruker Employee

HSDC files are usually so large that they are unmanageable when exported to ASCII.  It is usually much more efficient to directly read the native NanoScope files instead.  This is especially true if you plan on analyzing a large number of files.

Nanoscope files are composed of an ASCII header that contains information about the microscope configuration and scaling factors, followed by a binary section where the data is stored.  If you open a Nanoscope file with a text editor like notepad, you will see the header and you can get an idea of what is there.  There is a description of the header structure in the NanoScope software manual appendices.

Most modern analysis packages like MatLab, Python, Igor, LabView, etc. have simple functions to read binary data from a file.  

In Python it looks something like this:

with open(filename,"rb") as cc:
    cc.seek(DataOffset)
    dataA = DataScale * numpy.fromstring(cc.read(DataLength),numpy.int16).astype(float)/65536.0

 

In MatLab, something like:

cc=fopen('hsdc.000,'rb');
fseek(cc, DataOffset, -1);
dataA = DataScale * fread(cc, DataLength/2, 'int16');
fclose(cc);

 

In Igor, something like:

GBLoadWave /Q /B /A=dataA /T={16,4} /S=DataOffset /W=1 /U=DataLength "filename"
dataA=dataA*DataScale/65536.0


In all cases, the DataOffset, DataLength and DataScale for each channel comes from the ASCII header.  In notepad, that section of the header looks something like this:

\*Ciao HSDC list

\Data offset: 40960

\Data length: 5000000

 

Top 500 Contributor
1 Posts
Points 10
Answered (Not Verified) Jan replied on Thu, Jun 3 2010 6:00 AM
Suggested by Jan

Hi Cosmin,

SPIP can read HSDC files and provides batch facility to convert the individual recordings to binary or ASCII files.

SPIP can be downloaded from www.imagemet.com/download

If you need any assistance just let me know.

Best regards,

Jan

Jan F. Jorgensen
Image Metrology
jfj@imagemet.com

 

 

 

  • | Post Points: 10
Top 75 Contributor
12 Posts
Points 139
Peter replied on Thu, Jun 3 2010 7:19 AM

I agree that some sort of data processing in Python, MATLAB, etc. is a good way to process large amounts of data. The binary file structure may be slightly different depending on the Nanoscope software version you use, but it shouldn't be hard to extract the data. I have written some simple code in Scilab to extract CITS data from files produced by the 4.23 version of the Nanoscope software. You can get it here: http://www.nanotechnology.hu/~nemes/stm/

This could help you get started on extracting the data you need. Once exported to MATLAB, Scilab, etc. you have a lot of flexibility in plotting, handling your data.

 

Cheers,

         Peter

 

  • | Post Points: 12
Top 500 Contributor
2 Posts
Points 22

Thanks Bede and Peter,

I am trying to use Igor . The files I am working with contain two graphs - deflection vs time and z sensor vs time; in the file header it's specified that the DataOffset is 40960 and DataLength is 600000 for each graph.

I was able to read the file using the Igor comand Bede sugested, but I do not get the right time scale. When I export the graphs manualy with the Nanoscope, I get  two data rows for each graph - the y axis and x axis (time).

When I read the whole file in Igor I just get a single row - with the y data, first the deflection values, then the z sensor values. Any idea on how to extract the time scale ?

Thanks,

Cosmin

  • | Post Points: 12
Top 10 Contributor
280 Posts
Points 6,221
Bruker Employee

Hi Cosmin,

There are separate sections in the header for each data channel.  Each of the sections starts with a line "\*Ciao HSDC list".  You will find different Data offsets, lengths and scales for each channel.

I looked around and found an Igor procedure to read HSDC files!  

Here is a description of the procedure:

 

Igor function to read a Nanoscope High Speed Data Capture (HSDC) file and load the data into Igor waves with appropriate scaling.  The native Nanoscope file is read and the ASCII header is parsed to get scaling factors while the binary part is read directly, so no export is required.

See the comments at the beginning of the file for more details on usage.

This function was created by Ben Ohler in June 2006.  The code has been tested with Igor v5 and Nanoscope v7 and v8 HSDC files.

Please rename the extension from .txt to .ipf prior to using it

 OpenHSDC.txt

I think that will do what you need.  If you make improvements, please post them back here.

--Bede

Top 500 Contributor
2 Posts
Points 22

 

Great, the procedure it's working! - I am using Igor 6.0 and the HSDC files are generated with Nanoscope v 7.3.

I will try to modify the procedure so it will take a folder with HSDC files and load it into Igor all at once. I'll post it here when I am done.

Thanks,

Cosmin

  • | Post Points: 10
Page 1 of 1 (7 items) | RSS
Copyright (c) 2011 Bruker Instruments