idlastro / Miscellaneous (Non-Astronomy) Procedures: READCOL

[Source code]

NAME
READCOL
PURPOSE
Read a free-format ASCII file with columns of data into IDL vectors 
EXPLANATION
Lines of data not meeting the specified format (e.g. comments) are 
ignored.  By default, columns may be separated by commas or spaces.
Use READFMT to read a fixed-format ASCII file.   Use RDFLOAT for
much faster I/O (but less flexibility).    Use FORPRINT to write 
columns of data (inverse of READCOL).   
If you sure that all lines meet the specified format (excluding 
commented and SKIPed lines) then the speed for reading large files
can be significantly improved by setting the /QUICK keyword.
CALLING SEQUENCE
READCOL, name, v1, [ v2, v3, v4, v5, ...  v50 , COMMENT=, /NAN
    DELIMITER= ,FORMAT = , /DEBUG ,  /SILENT , SKIPLINE = , NUMLINE = 
    COUNT =, STRINGSKIP= 
INPUTS
NAME - Name of ASCII data file, scalar string.  
OPTIONAL INPUT KEYWORDS
FORMAT - scalar string containing a letter specifying an IDL type
        for each column of data to be read.  Allowed letters are 
        A - string data, B - byte, D - double precision, F- floating 
        point, I - short integer, L - longword, LL - 64 bit integer, 
        U - unsigned short integer, UL - unsigned long integer 
        Z - longword hexadecimal, and X - skip a column.
        Columns without a specified format are assumed to be floating 
        point.  Examples of valid values of FMT are
'A,B,I'        ;First column to read as a character string, then 
                1 column of byte data, 1 column integer data
'L,L,L,L'       ;Four columns will be read as longword arrays.
' '             ;All columns are floating point
If a FORMAT keyword string is not supplied, then all columns are 
assumed to be floating point.
/SILENT - Normally, READCOL will display each line that it skips over.
        If SILENT is set and non-zero then these messages will be 
        suppressed.
/DEBUG - If this keyword is non-zero, then additional information is
         printed as READCOL attempts to read and interpret the file.
COMMENT - single character specifying comment character.   Any line 
         beginning with this character will be skipped.   Default is
         no comment lines.
/COMPRESS - If set, then the file is assumed to be gzip compressed.
         The file is assumed to be compressed if it ends in '.gz'
DELIMITER - Character(s) specifying delimiter used to separate 
         columns.   Usually a single character but, e.g. delimiter=':,'
         specifies that either a colon or comma as a delimiter. 
         Set DELIM = string(9b) to read tab separated data
         The default delimiter is either a comma or a blank.
/NAN - if set, then an empty field will be read into a floating or 
         double numeric variable as NaN; by default an empty field is 
         converted to 0.0.
/PRESERVE_NULL - If set, then spaces are considered to be valid fields,
         useful if the columns contain missing data.   Note that between
         April and December 2006, /PRESERVE_NULL was the default.
/QUICK -  If set, then READCOL does not check that each individual line
         matches the supplied format.     This makes READCOL less 
         flexible but can provide a significant speed improvement when
         reading large files.                       
SKIPLINE - Scalar specifying number of lines to skip at the top of file
        before reading.   Default is to start at the first line.
NUMLINE - Scalar specifying number of lines in the file to read.  
        Default is to read the entire file
STRINGSKIP - will skip all lines that begin with the specified string.
        (Unlike COMMENT this can be more than 1 character.) Useful to 
        skip over comment lines.
OUTPUTS
V1,V2,V3,...V50 - IDL vectors to contain columns of data.
        Up to 50 columns may be read.  The type of the output vectors
        are as specified by FORMAT.
OPTIONAL OUTPUT KEYWORDS
COUNT - integer giving the number of valid lines actually read
NLINES - integer giving the total number of lines in the file 
         (as returned by FILE_LINES)
EXAMPLES
Each row in a file position.dat contains a star name and 6 columns
of data giving an RA and Dec in sexagesimal format.   Read into IDL 
variables.   (NOTE: The star names must not include the delimiter 
as a part of the name, no spaces or commas as default.)
IDL> FMT = 'A,I,I,F,I,I,F'
IDL> READCOL,'position.dat',F=FMT,name,hr,min,sec,deg,dmin,dsec  
The HR,MIN,DEG, and DMIN variables will be integer vectors.
Alternatively, all except the first column could be specified as
floating point.
IDL> READCOL,'position.dat',F='A',name,hr,min,sec,deg,dmin,dsec 
To read just the variables HR,MIN,SEC
IDL> READCOL,'position.dat',F='X,I,I,F',HR,MIN,SEC
RESTRICTIONS
This procedure is designed for generality and not for speed.
If a large ASCII file is to be read repeatedly, it may be worth
writing a specialized reader.
Columns to be read as strings must not contain the delimiter character
(i.e. commas or spaces by default).   Either change the default 
delimiter with the DELIMITER keyword, or use READFMT to read such files.
Numeric values are converted to specified format.  For example,
the value 0.13 read with an 'I' format will be converted to 0.
PROCEDURES CALLED
GETTOK(), STRNUMBER()
The version of STRNUMBER() must be after August 2006.
REVISION HISTORY
Written         W. Landsman                 November, 1988
 Added DELIMITER keyword  W. Landsman          Nov. 1999
Hexadecimal support added.  MRG, RITSS, 15 March 2000.
Default is comma or space delimiters as advertised   W.L. July 2001
Faster algorithm, use STRSPLIT if V5.3 or later  W.L.  May 2002
Accept null strings separated by delimiter ,e.g. ',,,'
Use SCOPE_VARFETCH instead of EXECUTE() for >V6.1  W.L. Jun 2005
Added compile_opt idl2   W. L.  July 2005
Added the NaN keyword   W. L      August 2006
Added /PRESERVE_NULL keyword  W.L.  January 2007
Assume since V5.6 (FILE_LINES available ) W.L. Nov 2007
Added COUNT output keyword  W.L.  Aug 2008
Added NLINES output keyword W.L.   Nov 2008
Added SKIPSTART keyword  Stephane Beland January 2008
Renamed SKIPSTART to STRINGSKIP to keep meaning of SKIP W.L. Feb 2008
Assume since V6.1, SCOPE_VARFETCH available W.L. July 2009
Read up to 40 columns W.L. Aug 2009
Use pointers instead of SCOPE_VARFETCH. Fixes bug with
IDL Workbench and runs 20% faster Douglas J. Marshall/W.L. Nov 2009
Recognize  LL, UL, and ULL data types, don't use 'val' output from 
    STRNUMBER()   W.L.  Feb 2010
Graceful return even if no valid lines are present D. Sahnow April 2010
Ability to read tab separated data WL April 2010
Free memory used by pointers  WL  July 2010
Added /QUICK keyword  WL  Sep 2010
Accept normal FORTRAN formats (e.g. F5.1) P. Noterdaeme/W.L Jan 2011
Add COMPRESS keyword, IDL 6 notation W. Landsman/J. Bailin   Feb 2011
Allow filename to be 1 element array W.Landsman/S.Antonille Apr 2011
Feb 2010 change caused errors when reading blanks as numbers. 
                   W.L. July 2012
Read up to 50 columns W.L.  March 2013
Assume a compressed file if it ends in '.gz'  W.L.  Oct 2015
Avoid error if more format codes than output variables W.L. April 2017