How to run a sequence of SQL queries and save the results?

Question

In other statistical programs, it's possible to create a log file that shows the output issued as a result of a command. Is it possible to do something similar in SQL?

In particular, I'd like to have a single .sql file with many queries and to then output each result to a text file.

I'm using PostgreSQL and Navicat.

Answer 1

plpgsql function and COPY

One way would be to put the SQL script into a plpgsql function, where you can write the individual return values to files with COPY and compile a report from intermediary results just like you need it.

This has additional effect that may or may not be desirable. Like, you can grant or revoke permission to the whole function to arbitrary roles. Read about SECURITY DEFINER in the manual. And the syntax will be verified when you save the function - however, only superficially (there are plans to change that in the future). More details in this answer on dba.SE.

Basic example:

CREATE OR REPLACE FUNCTION func()
  RETURNS void AS
$BODY$
BEGIN

COPY (SELECT * FROM tbl WHERE foo) TO '/path/to/my/file/tbl.csv';

COPY (SELECT * FROM tbl2 WHERE NOT bar) TO '/path/to/my/file/tbl2.csv';

END;
$BODY$
  LANGUAGE plpgsql;

Of course, you need to have the necessary privileges in the database and in the file system.
Call it from the shell:

psql mydb -c 'SELECT func();'

---

psql switching between meta commands and SQL

#!/bin/sh

BASEDIR='/var/lib/postgresql/outfiles/'

echo "
\\o $OUTDIR/file1.txt  \\\\\\\\ SELECT * FROM tbl1;
\\o $OUTDIR/file2.txt  \\\\\\\\ SELECT * FROM tbl2;
\\o $OUTDIR/file3.txt  \\\\\\\\ SELECT * FROM tbl3;" | psql event -p 5432 -t -A

That's right, 8 backslashes. Results from a double backslash that gets interpreted two times, so you have to double them two times.

I quote the manual about the meta-commands \o:

Saves future query results to the file filename or ...

and \\:

command must be either a command string that is completely parsable by the server (i.e., it contains no psql-specific features), or a single backslash command. Thus you cannot mix SQL and psql meta-commands with this option. To achieve that, you could pipe the string into psql, like this: echo '\x \\ SELECT * FROM foo;' | psql. (\\ is the separator meta-command.)

Answer 2

Don't know about navicat, but you can do it with psql. Check the various --echo-X command-line options and the \o command if you just want temporary output to a file.