How to configure a Mule flow to write oracle sql results to a file?

Question

Is there anyway I could configure, using the out of the box mule components, a flow that would write the result set of a select statement (oracle db) in a file?

I've tried numerous combination of transformers but only one did half of the job.

Here is the configuration XML:

    <?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:data-mapper="http://www.mulesoft.org/schema/mule/ee/data-mapper" xmlns:quartz="http://www.mulesoft.org/schema/mule/quartz" xmlns:wmq="http://www.mulesoft.org/schema/mule/ee/wmq" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns:jdbc-ee="http://www.mulesoft.org/schema/mule/ee/jdbc" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns:mulexml="http://www.mulesoft.org/schema/mule/xml" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
    xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.4.1"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/xml http://www.mulesoft.org/schema/mule/xml/current/mule-xml.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/ee/jdbc http://www.mulesoft.org/schema/mule/ee/jdbc/current/mule-jdbc-ee.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/ee/wmq http://www.mulesoft.org/schema/mule/ee/wmq/current/mule-wmq-ee.xsd
http://www.mulesoft.org/schema/mule/quartz http://www.mulesoft.org/schema/mule/quartz/current/mule-quartz.xsd
http://www.mulesoft.org/schema/mule/ee/data-mapper http://www.mulesoft.org/schema/mule/ee/data-mapper/current/mule-data-mapper.xsd">
    <jdbc-ee:oracle-data-source name="Oracle_Data_Source1" user="username" password="password" url="jdbc:oracle:thin:@//ip:1521/instance_name" transactionIsolation="READ_COMMITTED" doc:name="Oracle Data Source"/>
    <jdbc-ee:connector name="Database" dataSource-ref="Oracle_Data_Source1" validateConnections="true"  queryTimeout="-1" pollingFrequency="0" doc:name="Database" transactionPerMessage="false">
        <jdbc-ee:query key="selectTOP10" value="select CONTRACT_ID FROM LAM_CONTRACT where rownum&lt;11"/>
    </jdbc-ee:connector>
    <data-mapper:config name="map_to_map" transformationGraphPath="map_to_map.grf" doc:name="map_to_map"/>
    <file:connector name="File" writeToDirectory="D:\Test File Transfer\Destination" workFileNamePattern="test2.txt" autoDelete="true" outputAppend="true" streaming="true" validateConnections="true" doc:name="File"/>
    <file:endpoint path="D:\Test File Transfer\Destination" outputPattern="test2.txt" name="File1" responseTimeout="10000" doc:name="File"/>
    <flow name="oracle_to_fileFlow1" doc:name="oracle_to_fileFlow1">
        <jdbc-ee:inbound-endpoint  queryTimeout="-1" pollingFrequency="1000" doc:name="Database" connector-ref="Database" queryKey="selectTOP10"/>
        <logger level="INFO" doc:name="Logger" message="Result: #[payload]------------------------"/>
        <byte-array-to-string-transformer doc:name="Byte Array to String"/>
        <file:outbound-endpoint responseTimeout="10000" doc:name="File" ref="File1"/>
    </flow>
</mule>

I replaced the username and password for the db connection. If I run the flow, it will fill the text file like this:

{CONTRACT_ID=12}{CONTRACT_ID=13}{CONTRACT_ID=11}{CONTRACT_ID=14}{CONTRACT_ID=15}{CONTRACT_ID=9}{CONTRACT_ID=8}{CONTRACT_ID=7}... and so on.

What I need to do next is: 1) I would like to output the date in a tabular format (first row - column names(header), rest of the rows data)

2) I would like to be able to set/change column delimiters

3) I believe I have to use the Quartz component to be able to start the flow at a certain moment and run the query only once. I've tried to configure one, but I after spending couple of hours on google on how to do that I figured it would be better to ask here.

Can anyone please assist? I would like to add that I am a C#, .NET, SQL Server developer, so I really hope I don't have to learn java in order to do this with Mule.

P.S.: Although I have cursed for so long and repeatedly SSIS, I have to admit it rules compared to Mule - personal opinion only :).

Answer 1

1&2) If you have Mule EE (like in your example), this is not difficult. To do this in less than five minutes, you can use object-to-json-transformer, and then use DataMapper to create a transformation from JSON to CSV. You can choose a delimiter and add column headers to the output from the settings.

3) Assuming you can google for how to set a cron expression for Mule Quartz endpoint, you would first need an endpoint outside the flow:

<jdbc-ee:endpoint name="ep" queryKey="selectTOP10" connector-ref="Database" />

and the flow would be something like this:

<quartz:inbound-endpoint jobName="cronJobPoolTime" cronExpression="* * * * * ?">
    <quartz:endpoint-polling-job>
        <quartz:job-endpoint ref="ep"/>
    </quartz:endpoint-polling-job>
</quartz:inbound-endpoint>      
<json:object-to-json-transformer/>
<data-mapper:transform config-ref="json_to_csv"/>
<file:outbound-endpoint responseTimeout="10000" doc:name="File" ref="File1"/>

P.S.: My personal opinion is that using Mule professionally requires basic knowledge of Java and related technologies, such as Spring, Maven, etc. This is the stuff that the whole thing builds on.

Answer 2

For writting the resultset in a file simply remove < byte-array-to-string-transformer doc:name="Byte Array to String" /> and place Object to XML or Object to JSON Transformer ..