Recursive search through directory tree for files in date range - getting "Batch Recursion exceeds stack limits" error

Question 1

You have other problems besides recursion stack limits. The IF statement does not know how to compare dates. It only knows strings and numbers. In order to properly compare dates you must reformat the date to be in YYYYMMDD format.

Your timestamp information for use in the output file name should be collected all at once. Your existing code gets the hour at the beginning of the process, and the date, minutes and seconds at the end of the process. Not good. There could be a significant time gap between the start and end times.

Batch has two types of recursion errors:

1) Only 31 SETLOCAL levels within one CALL level.

2) A variable number of recursive CALLs allowed, depending on Windows version, machine memory, ...

There is no way to increase the size of the stack. If you get a recursion error, you must look for ways to reduce the amount of recursion. In your case, you can simply let FOR /R do all the recursion for you!

I modified the script such that the list of files is generated if a 4th argument is passed. I also incorporated the timestamp into file list file name.

The code assumes your machine's file date/time value starts with MM/DD/YYYY. The code will have to be modified if it does not.

@echo off
setlocal enableDelayedExpansion

REM This program takes three parameters <starting directory> <startdate> <enddate>
REM The startdate and endate should be in format: mm/dd/yyyy
REM The program will recursively look through all directories and sub-directories
REM from the given <starting directory> and count up the number of files written
REM within the date range from <startdate> until <endate>
REM It will then write out a RetentionReport_<date>_<time>.txt file that lists
REM one line showing the <startdate> <enddate> and the # of files found.

REM If you don't pass in all three arguments it will let you know and then exit.

REM You need to set your TESTDIR below to a hardpath location for the writing
REM of the Reports

REM If you want to print out a .tmp file that lists the files counted for the
REM period given then you can pass in a 4th argument with any value

if "%~3" == "" (
  echo Please pass in arguments for starting directory, startdate, and enddate.
  echo startdate and endate should be in the format mm/dd/yyyy
  exit/b
)

set "TAB=   "
set "TESTDIR=D:\TEST\"

set "startdate=%~2"
set "start=%startdate:~-4%%startdate:~0,2%%startdate:~3,2%"
set "enddate=%~3"
set "end=%enddate:~-4%%enddate:~0,2%%enddate:~3,2%"

set "timestamp=%date:~-4%%date:~-10,2%%date:~-7,2%_%time:~0,2%%time:~3,2%%time:~6,2%"
set "timestamp=%timestamp: =0%"

for /r "%~1" %%F in (*) do (
  set "dt=%%~tF"
  set "dt=!dt:~6,4!!dt:~0,2!!dt:~3,2!"
  if !dt! geq %start% if !dt! leq %end% (
    if "%~4" neq "" (echo %%~tF   %%F) >>"%TESTDIR%MONTH_files_%timestamp%.tmp"
    set /a cnt+=1
  )
)

(echo %startdate%%TAB%%enddate%%TAB%%cnt%) >>"%testdir%RetentionReport_%timestamp%.txt"

Question 2

You problem is caused by calling :listFolder within :listFolder.

@echo off
setlocal

REM This program takes three parameters <starting directory> <startdate> <enddate>
REM The startdate and endate should be in format: mm/dd/yyyy
REM The program will recursively look through all directories and sub-directories
REM from the given <starting directory> and count up the number of files written
REM within the date range from <startdate> until <endate>
REM It will then write out a RetentionReport_<date>_<time>.txt file that lists
REM one line showing the <startdate> <enddate> and the # of files found.

REM If you don't pass in all three arguments it will let you know and then exit.

REM You need to set your TESTDIR below to a hardpath location for the writing
REM of temporary files and writing of the Reports

REM There is one .tmp file created during processing and then deleted.
REM To prevent the .tmp file being deleted you can comment out the second
REM instance of this line by adding a REM in front of it:
REM if exist %TESTDIR%*.tmp del %TESTDIR%*.tmp

REM If you want to print out a .tmp file that lists the files counted for the
REM period given then you could remove the REM from in front of the following
REM line in the below code: echo %%~tF   %%F>> %TESTDIR%MONTH_files.tmp


set "startdate=%~2"
set "enddate=%~3"
IF DEFINED startdate IF DEFINED enddate GOTO parmsok
echo Please pass in arguments for starting directory, startdate, and enddate.
echo startdate and endate should be in the format mm/dd/yyyy
GOTO :eof

:parmsok
CALL :convdate startdate "%startdate%"
CALL :convdate enddate "%enddate%"
set "TAB=   "
set /a MONTHTOTAL=0
set /a GRANDTOTAL=0
set "TESTDIR=C:\TEST\"

if exist %TESTDIR%*.tmp del %TESTDIR%*.tmp
pushd %1
REM call :run >NUL
call :run
popd
FOR /F %%i IN (%TESTDIR%temp_TotalByDir.tmp) DO set /a MONTHTOTAL+=%%i
set hr=%time:~0,2%
set hr=%hr: =0%
echo %startdate%%TAB%%enddate%%TAB%%MONTHTOTAL% >> "%TESTDIR%RetentionReport_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%.txt"
ECHO %grandtotal%
if exist %TESTDIR%*.tmp del %TESTDIR%*.tmp
GOTO :eof 

:run
for /d /r . %%T in (.) do (
  pushd "%%T"
  IF NOT ERRORLEVEL 1 (
   call :listFolder
   POPD
  )
)
GOTO :eof

:listFolder
set /a ADD=0

for %%F in (*) do (
  CALL :convdate filedate "%%~tF"
  CALL :compdate
  IF DEFINED inrange (
  REM  echo %%~tF   %%F>> %TESTDIR%MONTH_files.txt
    set /a ADD+=1
    SET /a GRANDTOTAL+=1
  )
)
echo %ADD% >> %TESTDIR%temp_TotalByDir.tmp

GOTO :EOF

:: Convert date in %2 to yyyymmdd in %1

:convdate
SET "$1=%~2"
:: replace any space with 0
SET $1=%$1: =0%
:: convert date format. I use dd/mm/yyyy.
SET %1=%$1:~6,4%%$1:~3,2%%$1:~0,2%
:: version for mm/dd/yyyy.
REM SET %1=%$1:~3,2%%$1:~6,4%%$1:~0,2%
GOTO :EOF

:: Set Inrange iff date is in range

:compdate
SET "inrange="
if %filedate% GEQ %startdate% if %filedate% LEQ %enddate% SET inrange=Y
GOTO :eof

Interesting exercise.

A few points:

Delayed expansion is not necessary.

Assign the daterange variables. Only if both exist proceed else error message and exit.

Set common variables. Note the syntax set "var=string" is designed to ensure that trailing spaces on the line are not included in the value assigned. The set /a syntax to assign an numeric value is immune to trailing spaces.

PUSHD your target directory; POPD after processing to return to your original.

I removed the >nul from the call :run to allow kbd interrupt

The syntax `set "var=%var: =0%" replaces spaces with zeroes.

I moved the setting of hr to just before the RetentionReport was generated in case the run-time caused the hour to change.

Your original (optional) filelist was written to a .tmp file which would have been deleted, so I changed it to a .txt file.

for /d /r does a recursive directory-name scan, including the current directory in the form shown.

if pushd fails (it did on directorynames containing ! with delayedexpansion) then it is ineffective, so only if errorlevel is 0 should the call and popd be executed.

listfolder examines exactly one directory - the current.

The dates need to be converted to yyyymmdd form for proper comparison. This need to be performed on the supplied dates just the once (no point in perpetually re-converting them within the loop) which is done at the start.

The main use for delayedexpansion is to access the value of a variable where the value chages within a loop; however, the if defined test (and if exist and if errorlevel n) acts on the run-time value, not the parse-time value. Hence this method avoids the use of delayedexpansion - for a demonstration, if nothing else.

Final remarks: Executing an xcopy /L /D with an appropriate date and to an empty directory would list the files that were generated on and after the selected date. Therefore, running XCOPY /L /D to an empty directory for the two dates could be coaxed to produce a filecount; #earlier - #later = #between.

The count of files (=lines) in MONTH_files.txt would be equal to the GRANDTOTAL (and MONTHTOTAL) - but would be a good idea to delete MONTH_files.txt at the start (no longer a .tmp file...)

Question 3

The number of folders have no relation with the recursive level. If each directory level is processed with a recursive call, the maximum deep of recursive calls should be 10 or 11 in a very large tree. I suggest you to start with the code below as base and modify it for your needs:

@echo off
call :treeProcess
goto :eof

:treeProcess
rem Do whatever you want here over the files of this subdir, for example:
copy *.* C:\dest\dir
for /D %%d in (*) do (
    cd %%d
    call :treeProcess
    cd ..
)
exit /b

EDIT

I reviewed your code and the only strange point is the endlocal command placed below :run label. I think that at this point all previously defined variables are released, with unpredictable results. This is a simpler version of your code that I think should correctly work:

@echo off
setlocal

set "TAB=   "
set "MONTHTOTAL=0"
set hr=%time:~0,2%
if "%hr:~0,1%" equ " " set hr=0%hr:~1,1%
set "TESTDIR=C:\TEST\"

set "startdate=%~2"
set "enddate=%~3"

if exist %TESTDIR%*.tmp del %TESTDIR%*.tmp
call :run >nul
FOR /F %%i IN (%TESTDIR%temp_TotalByDir.tmp) DO set /a MONTHTOTAL+=%%i
echo %startdate%%TAB%%enddate%%TAB%%MONTHTOTAL% >> %TESTDIR%RetentionReport_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%.txt
if exist %TESTDIR%*.tmp del %TESTDIR%*.tmp
exit /b

:run

:listFolder

set "ADD=0"

for %%F in (*) do ( 
   if %%~tF GEQ %startdate% (
      if %%~tF LEQ %enddate% (
         REM echo %%~tF   %%F>> %TESTDIR%MONTH_files.tmp
         set /a ADD+=1
      )
   )
)

echo %ADD% >> %TESTDIR%temp_TotalByDir.tmp

for /d %%F in (*) do (
  cd "%%F"
  call :listFolder
  cd ..
)
exit /b