4CCE1PHC Physical Computing - Assignment: Machine Learning with a Neural Network
Assignment: Machine Learning with a Neural Network
1 Introduction
As a C programmer (and an engineer), you will frequently be asked to use your specialist skills to contribute functional code as a smaller part of a larger project. In this assignment, you will write code to implement components and functions of a larger C program that implements a simple, artificial neural network (ANN), a kind of machine learning system. Note that, you do not need any in-depth knowledge of ANNs or any other machine learning technique to complete this assignment.
2 Background
Artificial neural networks are kind of model often used in artificial intelligence for allowing machines to learn how to solve problems. They work by taking data about a problem, and training until they can solve it accurately. Originally, they were developed to imitate how the human brain solves problems, but since then ANNs of many shapes and sizes have been developed, including very large ones (so-called deep learning models). A conceptual diagram of the ANN you will be working with is shown in Fig. 1.
Figure 1: Single hidden layer neural network. The ANN works by propagating signals from left to right. Each big circle represents a “neuron” and takes a set of numbers as inputs (marked by the arrows going into the neurons in the diagram), performs some simple mathematical operations on them, and produces an output (marked by the arrows going out of the neurons). This output may be fed as the input to another neuron, or it may form the output of the whole network.
Table 1: Truth table of XOR logical operator. The neural network program should learn to predict the correct output, given each of these possible inputs.
3 Preparation
To complete the assignment, you will need the following files:
- in.csv
- out.csv
- ann.c
- libann.h
- libann.c
- k0000000.c
Download these from KEATS and save them to your computer. Important in.csv and out.csv are comma separated value files, a simple format in which values are stored as plain text. When downloading these, make sure you do not accidentally save them as another format (e.g., Excel, ODS). To test that you have downloaded them correctly, open the file in a plain text editor and check that you can read the values. For instance, when you open in.csv in a text editor you should see the following:
0 ,0
0 ,1
1 ,0
1 ,1Use the following commands to compile the program:
$ gcc libann . c k0000000 . c ann . c -o ann
$ ./ annYou should find that it compiles without errors.1 If you have completed this correctly, you should see the following output in your terminal:
Important In the following, you will be asked to implement functions in k0000000.c to create a working ANN. When completing the tasks, do not change the number or type of parameters used in the functions nor the type of their return values. Do not add or remove any functions. Do not add or remove inclusion of any header files or libraries. In short, do not modify any file, other than k0000000.c and only edit the code at the locations indicated in the tasks. You are encouraged to comment your code to aid understanding for the markers.
4 Tasks
Complete the following tasks.
- The first task is to write a function to read in the data required for training the network, or report
an error if the data cannot be found.
To complete this part of the assignment, find and modify the following lines in the template file:
/* - - - - - - - - - - - - - - - Begin Answer to Task 1 Here - - - - - - - - - - - - */ return 0; /* - - - - - - - - - - - - - - - End Answer to Task 1 Here - - - - - - - - - - - - - - */The function should take pointers to the arrays trIn and trOut, and the strings input_data_file and output_data_file as arguments. Implement code that reads the training data from the files in.csv and out.csv and stores them in the arrays trIn and trOut. The first index of the array trIn should correspond to the index of the data point (i.e., row of in.csv) and the second should correspond to the index of the input (i.e., Input 1 or Input 2 ). The function should return one and exit immediately if any errors are encountered when reading the files, or zero otherwise. [6 marks]
- The next task is to implement the basic building blocks of the neural network: the neurons.
To complete this part of the assignment, find and modify the following lines in the template file:
/* - - - - - - - - - - - - - - - Begin Answer to Task 2 Here - - - - - - - - - - - - */ return 0; /* - - - - - - - - - - - - - - - End Answer to Task 2 Here - - - - - - - - - - - - - - */Implement the function neuron() to compute the output of a neuron z from its inputs xi , using the following steps. Step 1. Each of the inputs xi should be multiplied by a weighting factor wi and added together according to the equation where b is a constant (called the bias term) and d is the number of inputs.
Step 2. The resultant value a should be passed through another function (called the activation function) to compute the output [6 marks]
- The final task is to write code to train the neural network on the data.
To complete this part of the assignment, find and modify the following lines in the template file:
/* - - - - - - - - - - - - - - - Begin Answer to Task 3 Here - - - - - - - - - - - - */ error = update_network ( trIn , trOut , DeltaWeightIH , DeltaWeightHO , DeltaBiasIH , DeltaBiasHO , WeightIH , WeightHO , biasIH , biasHO , DeltaO , DeltaH , SumDOW , eta , alpha , inds [0]); /* - - - - - - - - - - - - - - - End Answer to Task 3 Here - - - - - - - - - - - - - - */
Neural networks of the kind implemented here learn by iteration. At each step, the training data is fed to the network, and the network updates itself by (i) trying to predict the output data from the input data, and (ii) correcting the errors it makes. It can take many iterations before a good prediction is found. In the template code, training is performed iteratively in the function learn(). This function has been partially implemented for you.
In the template code, the function update_network() is used to update the network. This function takes as input (i) the current network state as parameters (i.e., weights, bias terms, etc) and learning parameters (i.e., eta and alpha). Its last argument p is the index of the training pattern (i.e., row of trIn) that the network should use for testing its predictions. Learning usually works best if this index is randomly shuffled after each training step (i.e., the network is given the data in random order). The template code provides the function shuffle_index() to compute a set of indices that can be used to access the training data array in random order.
