Using SAS Macro to pipe a list of filenames from a Windows directory
https://stackoverflow.com/questions/1409543
italiano
english
français
española
中国
日本の
العربية
Deutsch
한국어
Português
RussianQuestion
I am trying to amend the macro below to accept a macro parameter as the 'location' argument for a dir command. However I cannot get it to resolve correctly due to the nested quotes issue. Using %str(%') does not work, neither do quoting functions for some reason.
The macro will work fine when the filepath has no spaces (eg C:\temp\withnospace) as the middle quotes aren't needed. However I need this macro to work for filepaths with spaces (eg 'C:\temp\with space\').
Please help!
%macro get_filenames(location)
filename pipedir pipe "dir &location. /b " lrecl=32767;
data filenames;
infile pipedir truncover;
input line $char1000.;
run;
%mend;
%get_filenames(C:\temp\) /* works */
%get_filenames('C:\temp\with space') /* doesnt work */
Solution
Here's another way of achieving the same result without needing to use a PIPE.
%macro get_filenames(location);
filename _dir_ "%bquote(&location.)";
data filenames(keep=memname);
handle=dopen( '_dir_' );
if handle > 0 then do;
count=dnum(handle);
do i=1 to count;
memname=dread(handle,i);
output filenames;
end;
end;
rc=dclose(handle);
run;
filename _dir_ clear;
%mend;
%get_filenames(C:\temp\);
%get_filenames(C:\temp\with space);
%get_filenames(%bquote(C:\temp\with'singlequote));
OTHER TIPS
Make the following several changes and your code will work.
%macro get_filenames(location); %*--(1)--*;
filename pipedir pipe "dir ""%unquote(&location)"" /b" lrecl=32767; %*--(2)--*;
data filenames;
infile pipedir truncover;
input filename $char1000.;
put filename=;
run;
filename pipedir clear; %*--(3)--*;
%mend;
%get_filenames(d:\)
%get_filenames(d:\your dir) %*--(4)--*;
(1) End the %macro statement with a semi-colon;
(2) Surround the macro variable resolution with doubled-up double quotes and %unquote;
(3) Release the file handle by clearing it; and
(4) Don't single quote your input parameter. macro quote instead, if necessary.
Based on the last sample on this page, instead of the filename statement, try
%let filrf=pipedir;
%let rc=%sysfunc(filename(filrf,%bquote(dir "&location" /b),pipe));
and call the macro without using quotes:
%get_filenames(c:\temp\with spaces);
I also tried macro quoting, but couldn't get it to work.
here's a quick macro to pull windows-based directory listings into a sas data set.
%macro DirList(dir);
/* %if &SUBDIR eq %then %let subdir=/s; */ /*** &SUBDIR not defined ****/
filename dirpipe pipe "dir &DIR.\*.* /s /-c";
data dir_list(label="Directory Listing [&DIR.]" drop=re_: _line_ date time);
format Path
File $250.
ModDT datetime19.
Size 16.
_line_ $32000. ;
if _N_ = 1 then do;
re_path=prxparse("/Directory of (.+)/");
re_subd=prxparse("/(\d\d\/\d\d\/\d\d\d\d)\s+(\d\d:\d\d [A|P]M)\s+\s+(\S.*)/");
re_file=prxparse("/(\d\d\/\d\d\/\d\d\d\d)\s+(\d\d:\d\d [A|P]M)\s+(\d+)\s+(\S.*)/");
retain re_: path;
end;
infile dirpipe lrecl=32000; input; _line_ = _infile_;
if lengthn(_line_)=0 then delete;
else
if prxmatch(re_path, _line_) then do;
path=prxposn(re_path, 1, _line_);
end;
else
if prxmatch(re_subd, _line_) then do;
date=input(prxposn(re_subd, 1, _line_), mmddyy10.);
time=input(prxposn(re_subd, 2, _line_), time6.);
ModDT=dhms(date, 0, 0, time);
File=prxposn(re_subd, 3, _line_);
size = .D; /*mark subdirectory records*/
if file not in ('.', '..') then output;
end;
else
if prxmatch(re_file, _line_) then do;
date=input(prxposn(re_file, 1, _line_), mmddyy10.);
time=input(prxposn(re_file, 2, _line_), time6.);
ModDT=dhms(date, 0, 0, time);
size=input(prxposn(re_file, 3, _line_), 16.);
file=prxposn(re_file, 4, _line_);
output;
end;
run;
filename dirpipe clear;
%mend;
and here's how they get called
%dirlist(c:); %dirlist(c:\temp);
notice there is no trailing backslash when specifying the base directory. C: not C:\.
it works for me if i call the original macro this way
%get_filenames(""C:\Program Files"")
of course i had to add the semicolon at the end of the %macro statement.
if your directory contains a comma, bad things happen. to fix, use the %str() macro
%get_filenames(%str(C:\temp\comma, fail))
Here's a pure macro code version. It also allows you to specify that you only want to know about files (and not folders) and lets you specify a basic filter. It returns the list of files in a delimited format but you can easily insert these into a dataset using SQL insert if you wanted to (example included but not tested - no SAS access atm). It can be called from anywhere - within another macro, a dataset, an sql statement... wherever. Just add these two macros to your macro autocall library and you're right to go.
There are 2 macros below. The %isdir macro is required by the %file_list macro. The macros are a bit larger and more complex than the above but they are MUCH more flexible. Plus they provide error checking.
/******************************************************************************
** PROGRAM: ISDIR.SAS
**
** DESCRIPTION: DETERMINES IF THE SPECIFIED PATH EXISTS OR NOT.
** RETURNS: 0 IF THE PATH DOES NOT EXIST OR COULD NOT BE OPENED.
** 1 IF THE PATH EXISTS AND CAN BE OPENED.
**
** PARAMETERS: iPath: THE FULL PATH TO EXAMINE. NOTE THAT / AND \ ARE TREATED
** THE SAME SO &SASDIR/COMMON/MACROS IS THE SAME AS
** &SASDIR\COMMON\MACROS.
**
******************************************************************************/
%macro isDir(iPath=,iQuiet=1);
%local result dname;
%let result = 0;
%if %sysfunc(filename(dname,&iPath)) eq 0 %then %do;
%if %sysfunc(dopen(&dname)) %then %do;
%let result = 1;
%end;
%else %if not &iQuiet %then %do;
%put ERROR: ISDIR: %sysfunc(sysmsg());
%end;
%end;
%else %if not &iQuiet %then %do;
%put ERROR: ISDIR: %sysfunc(sysmsg());
%end;
&result
%mend;
%put %isDir(iPath=&sasdir/common/macros);
%put %isDir(iPath=&sasdir/kxjfdkebnefe);
%put %isDir(iPath=&sasdir/kxjfdkebnefe, iQuiet=0);
%put %isDir(iPath=c:\temp);
/******************************************************************************
** PROGRAM: FILE_LIST.SAS
**
** DESCRIPTION: RETURNS THE LIST OF FILES IN A DIRECTORY SEPERATED BY THE
** SPECIFIED DELIMITER. RETURNS AN EMPTY STRING IF THE THE
** DIRECTORY CAN'T BE READ OR DOES NOT EXIST.
**
** PARAMETERS: iPath : THE FULL PATH TO EXAMINE. NOTE THAT / AND \ ARE
** TREATED THE SAME SO &SASDIR/COMMON/MACROS IS THE
** SAME AS &SASDIR\COMMON\MACROS. WORKS WITH BOTH UNIX
** AND WINDOWS.
** iFilter : SPECIFY A BASIC FILTER TO THE FILENAMES, NO REGULAR
** EXPRESSIONS OR WILDCARDS.
** iFiles_only: 0=RETURN FILES AND FOLDERS
** 1=RETURN FILES ONLY.
** iDelimiter : SPECIFY THE DELIMITER TO SEPERATE THE RESULTS BY.
******************************************************************************/
/*
** TODO: DOESNT CATER FOR MACRO CHARS IN FILENAMES. FIX SOMETIME.
** TODO: IMPROVE THE FILTER. JUST A SIMPLE IF STATEMENT AT THE MOMENT.
*/
%macro file_list(iPath=, iFilter=, iFiles_only=0, iDelimiter=|);
%local result did dname cnt num_members filename;
%let result=;
%if %sysfunc(filename(dname,&iPath)) eq 0 %then %do;
%let did = %sysfunc(dopen(&dname));
%let num_members = %sysfunc(dnum(&did));
%do cnt=1 %to &num_members;
%let filename = %sysfunc(dread(&did,&cnt));
%if "&filename" ne "" %then %do;
%if &iFiles_only %then %do;
%if not %isDir(iPath=&iPath/&filename) %then %do;
%if "&iFilter" ne "" %then %do;
%if %index(%lowcase(&filename),%lowcase(&iFilter)) %then %do;
%let result = &result%str(&iDelimiter)&filename;
%end;
%end;
%else %do;
%let result = &result%str(&iDelimiter)&filename;
%end;
%end;
%end;
%else %do;
%if "&iFilter" ne "" %then %do;
%if %index(%lowcase(&filename),%lowcase(&iFilter)) %then %do;
%let result = &result%str(&iDelimiter)&filename;
%end;
%end;
%else %do;
%let result = &result%str(&iDelimiter)&filename;
%end;
%end;
%end;
%else %do;
%put ERROR: (CMN_MAC.FILE_LIST) FILE CANNOT BE READ.;
%put %sysfunc(sysmsg());
%end;
%end;
%end;
%else %do;
%put ERROR: (CMN_MAC.FILE_LIST) PATH DOES NOT EXIST OR CANNOT BE OPENED.;
%put %sysfunc(sysmsg());
%end;
/*
** RETURN THE RESULT. TRIM THE LEADING DELIMITER OFF THE FRONT OF THE RESULTS.
*/
%if "&result" ne "" %then %do;
%substr(&result,2)
%end;
%mend;
**
** EXAMPLES - HAVENT TESTED THE LAST TWO YET BUT THEY SHOULD WORK IF SYNTAX IS CORRECT
*;
%put %file_list(iPath=c:\temp);
%put %file_list(iPath=c:\xxdffsds);
%put %file_list(iPath=c:\rob\SASDev\, iFilter=a);
%put %file_list(iPath=c:\rob\SASDev\,iFiles_only=1);
%put %file_list(iPath=/tmp/unix_sasdir,iFiles_only=1);
data x;
file_list = "%file_list(iPath=c:\temp)";
run;
proc sql noprint;
insert into my_table values ("%file_list(iPath=c:\temp,iDelimiter=%str(","))");
quit;
Here's one that unscrambles the order of quoting and unquoting:
%let command =%unquote(%str(%')dir "&baseDir.data\*txt"%str(%'));
filename datain pipe &command;
where macro variable basedir can contain spaces and so can the filenames.
This combination of
%unquote and %str(%') is a frequently occuring macro idiom.
"what if I have single quote in my dir?"
Handling this situation requires a macro quoting function, such as %bquote();
Continuing the example above, this:
%let command =%unquote(%str(%')dir "%bquote(&baseDir.data\*txt)"%str(%'));
should do it.
To avoid infinite iterations of this kind of question, look at Ian Whitlock's paper, A Serious Look at Macro Quoting, which is available here;
There are (many) others, but this is the most widely cited. A little note: anything by Ian Whitlock is probably worthwhile. He writes clearly and his understanding of SAS issues is awesome.
We use this little macro
%macro getdir(dir=,redirect=, switch=);
options noxwait xsync;
%if %length(&switch)=0 %then %let switch=b;
data _null_;
xcmd='dir "' || "&dir" || '"' || "/&switch " || ">" || "&redirect";
put 'generated the following command: ' xcmd=;
rc=system(xcmd);
put 'result code of above command: ' rc=;
run;
%mend getdir;
Sample Call
%getdir(dir=c:\temp\,redirect=c:\temp\dir.txt) *run;
If you run in batch and don't have the option noxwait xsync the job will hang on the server waiting for an operator response.
