Encog Neural Net : Normalization

Question 1

Here are few resources ,where you can get more detailed information about normalization and denormalization using ENCOG framework.

These great e-books written by Jeff Heaton himself, 1. Programming Neural Networks with Encog3 in C#, 2nd Edition by Heaton, Jeff (Oct 2, 2011) 2.Introduction to Neural Networks for C#, 2nd Edition by Jeff Heaton (Oct 2, 2008) These are must have ebooks for ENCOG users.

You can also have a look at pluralsight course on "Introduction to Machine learning with ENCOG", this also includes few examples of normalization and denormalization.

Now regarding your queries :"It is not clear here though how to take the AnalystNormalizeCSV and convert it to an MLDataSet to actually use it."

well you can use AnalystNormalizeCSV to normalize your training file. And then you can use LoadCSV2Memory of EncogUtility class to load the normalized training file to get the ML DataSet. Here is a sample code in C#,

var trainingSet = EncogUtility.LoadCSV2Memory(Config.NormalizedTrainingFile.ToString(), network.InputCount, network.OutputCount,true, CSVFormat.English,false);

it takes the normalized training file as first parameter, network input neuron count as second, network output neuron count as third, fourth parameter is boolean if you have header in your csv file, then you can mention the format as the fifth parameter, and sixth parameter is for significance.

so once you have this dataset in memory, you can use it for training. Similar approach can be taken in cross validation and evaluation step also.

Regarding denormalization, you can first persist the analyst file, and later you can use analyst file to denormalize individual columns also. For example :

var denormlizedOutput = analyst.Script.Normalize.NormalizedFields[index].DeNormalize(item.Input[index]);

Similar approach can be used in denormalizing fields to get class labels also. For example

var predictedClass = analyst.Script.Normalize.NormalizedFields[index].Classes[predictedClassInt].Name;

Question 2

The encog analyst is fantastic for normalizing data. It can take information stored in a CSV file and automatically determine the normalized fields and their type of encoding (including 1 of N equilateral encoding).

The only downside of this is that the logic is tightly coupled with the ReadCSV class.

Favouring extension as opposed to modification I decided to go about creating extension methods and alternative classes to create an analyst that would normalize a generic .NET dataset.

I also added a new test class which shows you how to use it (its very similar to the standard encog implementation).

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Collections.Generic;
using Encog.ML.Data.Market;
using Encog.ML.Data.Market.Loader;
using Encog.App.Analyst;
using Encog.App.Analyst.Wizard;
using EncogExtensions.Normalization; //<- This is my extension lib
using System.Data;
using System.Linq;     
[TestMethod]
    public void Normalize_Some_In_Memory_Data()
    {
        // Download some stock data
        List<LoadedMarketData> MarketData = new List<LoadedMarketData>();
        MarketData.AddRange(DownloadStockData("MSFT",TimeSpan.FromDays(10)));
        MarketData.AddRange(DownloadStockData("AAPL", TimeSpan.FromDays(10)));
        MarketData.AddRange(DownloadStockData("YHOO", TimeSpan.FromDays(10)));

        // Convert stock data to dataset using encog-extensions
        DataSet dataSet = new DataSet().Convert(MarketData, "Market DataSet");

        // use encog-extensions to normalize the dataset 
        var analyst = new EncogAnalyst();
        var wizard = new AnalystWizard(analyst);
        wizard.Wizard(dataSet);

        // DataSet Goes In... 2D Double Array Comes Out... 
        var normalizer = new AnalystNormalizeDataSet(analyst);
        var normalizedData = normalizer.Normalize(dataSet);

        // Assert data is not null and differs from original
        Assert.IsNotNull(normalizedData);
        Assert.AreNotEqual(normalizedData[0, 0], dataSet.Tables[0].Rows[0][0]);

    }

    private static List<LoadedMarketData> DownloadStockData(string stockTickerSymbol,TimeSpan timeSpan)
    {
        IList<MarketDataType> dataNeeded = new List<MarketDataType>();
        dataNeeded.Add(MarketDataType.AdjustedClose);
        dataNeeded.Add(MarketDataType.Close);
        dataNeeded.Add(MarketDataType.Open);
        dataNeeded.Add(MarketDataType.High);
        dataNeeded.Add(MarketDataType.Low);
        dataNeeded.Add(MarketDataType.Volume);

        List<LoadedMarketData> MarketData =
            new YahooFinanceLoader().Load(
                new TickerSymbol(stockTickerSymbol),
                dataNeeded,
                DateTime.Now.Subtract(timeSpan),
                DateTime.Now).ToList();

        return MarketData;
    }