CSSE2310/CSSE7231 Computer Systems Principles and Programming Assignment 3: testuqwordladder
CSSE2310/CSSE7231 — Semester 2, 2023
Assignment 3 (version 1.1 – 18 September 2023)
Marks: 75 (for CSSE2310), 85 (for CSSE7231)
Weighting: 15%
Due: 4:00pm Thursday 5th October, 2023
Introduction 1
The goal of this assignment is to demonstrate your skills and ability in fundamental process management and 2 communication concepts, and to further develop your C programming skills with a moderately complex program. 3 You are to create a program called testuqwordladder that creates and manages communicating collections 4 of processes that test a uqwordladder program (from assignment one) according to a job specification file that 5 lists tests to be run. For various test cases, your program will, if required, generate the expected output from 6 a known good program (good-uqwordladder) and then run the tests with a sample program (the program 7 under test) to compare their standard outputs, standard errors and exit statuses and report the results. The 8 assignment will also test your ability to code to a programming style guide and to use a revision control system 9 appropriately.1675 10 CSSE7231 students will be required to implement additional functionality for full marks. 11
Student Conduct 12
This section is unchanged from assignment one – but you should remind yourself of the referencing requirements. 13
This is an individual assignment. You should feel free to discuss general aspects of C programming 14 and the assignment specification with fellow students, including on the discussion forum. In general, questions 15 like “How should the program behave if ⟨this happens⟩?” would be safe, if they are seeking clarification on the 16 specification. 17
You must not actively help (or seek help from) other students or other people with the actual design, structure 18 and/or coding of your assignment solution. It is cheating to look at another student’s assignment code 19 and it is cheating to allow your code to be seen or shared in printed or electronic form by others. 20 All submitted code will be subject to automated checks for plagiarism and collusion. If we detect plagiarism or 21 collusion, formal misconduct actions will be initiated against you, and those you cheated with. That’s right, if 22 you share your code with a friend, even inadvertently, then both of you are in trouble. Do not post your 23 code to a public place such as the course discussion forum or a public code repository. (Code in private posts 24 to the discussion forum is permitted.) You must assume that some students in the course may have very long 25 extensions so do not post your code to any public repository until at least three months after the result release 26 date for the course (or check with the course coordinator if you wish to post it sooner). Do not allow others to 27 access your computer – you must keep your code secure. Never leave your work unattended. 28
You must follow the following code referencing rules for all code committed to your SVN repository 29
(not just the version that you submit):
Code Origin
Usage/Referencing
|
Code provided to you in writing this semester by CSSE2310/7231 teaching staff (e.g. code hosted on Black- board, found in /local/courses/csse2310/resources on moss, posted on the discussion forum by teaching staff, pro- vided in Ed Lessons, or shown in class). |
May be used freely without reference. (You must be able to point to the source if queried about it – so you may find it easier to reference the code.) |
|
Code you have personallywritten this semester for CSSE2310/7231 (e.g. code written for A1 reused in A3) – provided you have not shared or published it. |
May be used freely without reference. (This assumes that no reference was required for the original use.) |
Code Origin
Code examples found in man pages on moss.
Usage/Referencing
May be used provided you understand the code AND the source of the code is referenced in a comment adjacent to that code (in the required format – see the style guide). If such code is used without appropriate referencing then this will be considered misconduct.
Code you have personally written in a previous enrolment in this course or in another ITEE course and where that code has not been shared or published.
Code (in any programming language) that you have learned from or that you have taken inspiration from but have not copied1.
Code written by, modified by, or obtained from, or code based on code written by, modified by, or obtained from, a code generation tool (including any artificial intelligence tool) that you personally have interacted with, without the assistance of another person.
May be used provided you understand that code AND the source of the code is referenced in a comment ad- jacent to that code (in the required format) AND an ASCII text file (named toolHistory.txt) is included in your repository and with your submission that describes in detail how the tool was used. The file must be com- mitted to the repository at the same time as any code derived from such a tool. If such code is used with- out appropriate referencing and without inclusion of the toolHistory.txt file then this will be considered mis- conduct.
Other code – includes (but may not be limited to): code provided by teaching staff only in a previous offering of this course (e.g. previous assignment one solution); code from public or private repositories; code from websites or based on code from websites; code from textbooks; any code written or partially written or provided by or written with the assistance of someone else; and any code you have written that is available to other students.
May not be used. If the source of the code is referenced adjacent to the code then this will be considered code without academic merit (not misconduct) and will be removed from your assignment prior to marking (which may cause compilation to fail and zero marks to be awarded). Copied code without adjacent referencing will be considered misconduct and action will be taken.
You must not share this assignment specification with any person, organisation, website, etc. Up- 31 loading or otherwise providing the assignment specification or part of it to a third party including online tutorial 32 and contract cheating websites is considered misconduct. The university is aware of many of these sites and 33 many cooperate with us in misconduct investigations. You are permitted to post small extracts of this document 34 to the course Ed Discussion forum for the purposes of seeking or providing clarification on this specification. 35
The teaching staff will conduct interviews with a subset of students about their submissions, 36 for the purposes of establishing genuine authorship. If you write your own code, you have nothing to fear from 37 this process. If you legitimately use code from other sources (following the usage/referencing requirements in the 38 table above) then you are expected to understand that code. If you are not able to adequately explain the design 39 of your solution and/or adequately explain your submitted code (and/or earlier versions in your repository) 40 and/or be able to make simple modifications to it as requested at the interview, then your assignment mark will 41 be scaled down based on the level of understanding you are able to demonstrate and/or your submission may 42 be subject to a misconduct investigation where your interview responses form part of the evidence. Failure to 43 attend a scheduled interview will result in zero marks for the assignment. The use of referenced code in your 44 submission (particularly from artificial intelligence tools) is likely to increase the probability of your selection 45 for an interview. 46
In short – Don’t risk it! If you’re having trouble, seek help early from a member of the teaching staff. 47 Don’t be tempted to copy another student’s code or to use an online cheating service. Don’t help another 48 CSSE2310/7231 student with their code no matter how desperate they may be and no matter how close your 49 relationship. You should read and understand the statements on student misconduct in the course profile and 50 on the school website: https://eecs.uq.edu.au/current-students/guidelines-and-policies-students/ 51 student-conduct. 52
Specification – testuqwordladder 53
1675testuqwordladder reads a set of test jobs from a file whose name is provided as a command line argument. 54 (The format of this file is specified below.) testuqwordladder will, if required, generate the expected output 55 (standard output, standard error and exit status) of a known-good implementation (good-uqwordladder, found 56 in the user’s PATH2) and save these to files in a specified directory. Generation of these files will be required if 57
any of them are “out of date” with respect to the job specification file, i.e. the job specification file has been 58 modified more recently than one or more of the test output files, or if regeneration of all 1675output files has been 59 requested (via a command line argument). 60
After expected outputs have been generated (if necessary) then testuqwordladder will run each of the 61 test jobs using the program-under-test (named on the testuqwordladder command line, e.g. your solution to 62 assignment one) and compare its outputs (stdout and stderr) with the expected outputs and its exit status 63 with the expected exit status. 64
The comparison of the outputs must use the cmp program (found in the user’s PATH) with input piped from 65 the program-under-test. 66 Figure 1 illustrates the processes and pipes to be created by testuqwordladder for each test job (after 67 the expected outputs have been created if necessary). Three processes will be created (labelled A, B and C 68 in Figure 1): an instance of the program being tested (A), and two instances of cmp – one for comparing the 69 standard outputs of the two programs (B) and one for comparing the standard errors (C). This will require the 70 creation of two pipes (labelled 1 and 2 in Figure 1). The standard input for the program being tested will come 71 from a file whose name is specified in the job specification file – this contains sample user input for that test. 72 Full details of the required behaviour are provided below. 73
Figure 1: Processes (A to C) and pipes (1 to 2) to be created by testuqwordladder for each test job.1675
Command Line Arguments 74
testuqwordladder has two mandatory arguments – the name of the job specification file and the name of the 75 program being tested. There are three optional arguments which may appear in any order (but must appear 76
before the mandatory arguments).
Usage of the program and the meaning of the arguments are as follows:
./testuqwordladder [--diff N ] [--testdir dir ] [--rebuild] testfile
77 78
programname 79
-
--diff – this argument is for CSSE7231 functionality only (see page 9) – CSSE2310 students do not need 80 to check for or accept this argument – it will not be used in any tests for CSSE2310 students though it 81 must still be output in the usage message – see below. 1675 82
-
--testdir – if present, the associated value argument (dir) specifies the name of the directory where 83 the expected output files are to to be saved. If not present, the directory ./tmp is to be used. 84
-
--rebuild – if present, the expected output files must all be (re)generated, whether they are out of 85 date with respect to the job specification file or not. If not present, the expected output files are only 86 (re)generated if they don’t exist or are out of date with respect to the job specification file. 87
-
testfile – must be present and is the name of the file containing details of the tests to be run (referred 88 to in the remainder of this document as the job specification file). More details on the format of this file 89 are given below. The argument may not begin with --. (If a file’s name begins with -- then the user 90 must give the path to the file, e.g. ./--jobfile, to avoid a usage error.)1675 91
-
programname – must be present (after the job specification file name) and is the name of the program 92 being tested against good-uqwordladder. The name can be a relative or absolute pathname or otherwise 93 (if it doesn’t contain a slash ‘/’) is expected to be found on the user’s PATH. The argument may not 94 begin with --. (If a program’s name begins with -- then the user must give the path to the program, e.g. 95 ./--prog, to avoid a usage error.) 96
Prior to doing anything else, your testuqwordladder program must check the validity of the command line 97 arguments. If the program receives an invalid command line then it must print the message: 98
Usage: testuqwordladder [--diff N] [--testdir dir] [--rebuild] testfile programname 99 to standard error (with a following newline) and exit with an exit status of 7. 100
Invalid command lines include (but may not be limited to) any of the following: 101
-
arguments that begin with -- that are not either --diff or --testdir or --rebuild. 3 102
-
any of the --diff, --testdir or --rebuild option arguments is specified more than once on the command 103 line. (Note that it is not a usage error if the value argument associated with the --diff or --testdir 104 option arguments is one of these strings.) 105
-
an argument beginning with -- appears after the testfile or programname arguments 106
-
the programname and/or the testfile arguments are not specified on the command line 107
-
an unexpected argument is present 108
Checking whether the program name and/or the job specification file name are valid is not part of usage 109 checking. 110
Job Specification File 111
If the command line arguments are valid then testuqwordladder reads the job specification file listed on the 112 command line (the testfile argument). The whole file must be read and checked prior to any expected output 113 files being generated or any test jobs being run. 114
If testuqwordladder is unable to open the job specification file for reading then it must print the following 115 message to stderr (with a following newline) and exit with an exit status of 10:1675 116
testuqwordladder: Unable to open file "filename" 117 where filename is replaced by the name of the file (as given on the command line). (The double quotes around 118
the filename must be present in the output message.) 119 File Format 120
The job specification file provided to testuqwordladder is a text file, with one line per test job. The file may 121 contain any number of lines. 122 If testuqwordladder is able to open the job specification file for reading, then it should read and check all 123 of the test jobs in the file before generating any expected output files or starting any tests.1675 124 Lines in the job specification file beginning with the ‘#’ (hash) character are comments4, and are to be 125 ignored by testuqwordladder. Similarly, empty lines (i.e. with no characters before the newline) are to be 126 ignored.1675 127 All other lines are to be interpreted as job specifications, split over 2 or more separate fields delimited by 128
the tab character (’\t’ in C) as follows. The tab character is illustrated as indicate optional fields.
test-id input-file-name[ arg1[ arg2 ...]]
below. Square brackets ([ ]) 129 130
131
(Note that there is a extension to this format for CSSE7231 students – see page 9.) 132 The test-id field is a non-empty string that uniquely identifies the test and that contains no tab characters 133 or forward slashes (‘/’) . 134 The input-file-name field is the name of a file to be used as the standard input to the the test program 135 and to good-uqwordladder for that test. It may be an absolute pathname or relative to the current directory. 136 It must not be an empty field, and the file name can not contain tab characters. (A tab character will be 137 interpreted as the end of the file name and the start of the next field.) 138 The third and subsequent fields (if present) are command line arguments to be provided to the test program 139 and to good-uqwordladder when the test is run. These fields (if present) may be empty. Any number of 140 arguments may be specified. Arguments may not contain tab characters. (Tab characters are interpreted as 141 separators between arguments.) 142 Note that individual job specifications are independent – a given test job will always be run in the same way 143 independent of any other test jobs described in the file.1675 144
The tab character has special meaning in job specification files and will always be treated as a separator 145 between fields. It is not possible to specify a tab character as part of a test id, file name or an argument. See 146 the split_string() function described on page 11 for an easy way to split the tab-delimited job specifications. 147
Checking the Jobs 148
Each job listed in the job specification file must be checked in turn. Checks take place in the order described 149 below and if testuqwordladder must exit due to an error on 1675a line then no further lines are checked. Note 150 that for the error messages described in this section: 151
• Messages are printed to stderr and are followed by a newline. 152 • Line numbers in the job specification file are counted from 1 and include any comment or blank lines. 153 • Double quotes around file names in the error messages must be present. 154
If a job line is not syntactically valid then testuqwordladder must print the following message and exit 155 with an exit status of 13: 156
testuqwordladder: Line N of file "filename": syntax error 157
where N is replaced by the line number in the job specification file, and filename is replaced by the name of 158 the job specification file (as given on the command line). 159 A syntactically valid line will: 160 • contain at least one tab character, 161
• have at least one character before the first tab character (i.e. a non-empty test ID), 162 • have no forward slash (‘/’) characters before the first tab character (i.e. test IDs can’t contain ‘/’), and 163 • have at least one non-tab character after the first tab character (i.e. a non-empty input file name). 1675 164
If the line is syntactically valid but the test ID is the same as a test ID from a previous line then 165 testuqwordladder must print the following message and exit with an exit status of 1: 166
testuqwordladder: Line N of file "filename": duplicate job ID 167
where N is replaced by the line number in the job specification file (where the second use of the test ID is found), 168 and filename is replaced by the name of the job specification file. 169 If the line is syntactically valid and the test ID is not duplicated but the input file specified on the job line 170 is unable to be opened for reading, then testuqwordladder must print the following message and exit with an 171 exit status of 20: 172
testuqwordladder: Unable to open input file "inputfile" specified on line N of job file "fi1l73ename"
where inputfile is replaced by the name of the file specified on the job line, N is replaced by the line number 174 in the job specification file, and filename is replaced by the name of the job specification file (as given on the 175 command line). 176
If these checks are passed (the line is syntactically valid, the test ID is not a duplicate, and the input file 177 can be opened for reading) then testuqwordladder can move on to the next line in the job specification file. 178 If testuqwordladder reaches the end of the file and doesn’t find any job specifications (i.e. the file is empty 179 or only contains blank lines and/or comments) then testuqwordladder must print the following message and 180 exit with an exit status of 17: 181
testuqwordladder: Job spec file "filename" was empty 182
where filename is replaced by the name of the job specification file (as given on the command line). 183 The following are examples of valid jobfiles5 containing explanatory comments. Tab characters are illustrated
with the
symbol.
# Run test with no command line arguments and an immediate EOF on stdin
1.1
/dev/null
# Run test with an empty string as the only command line argument and immediate EOF on stdin
1.2
/dev/null
1 2 3 4
1 2 3
# Run two tests - the first specifies a starter word, the second specifies just a length
1.1
testfiles/cab-start --start cab
test2 input-file
--len 5
|
# Run four tests - with various combinations of arguments |
|
2.0
testfiles/2.in --start head
--length
5 # Blank line on the line above will be ignored |
1 2 3 4 5 6 7
Generating Expected Outputs 184
If all of the test jobs in the job specification file are valid then testuqwordladder must (re)generate the expected 185 outputs of each test, if necessary.1675 186 First, testuqwordladder must ensure that a test directory is created (or already exists) that will hold the 187 expected outputs. The directory name is given by the value associated with the --testdir command line 188 argument, or, if not present, then ./tmp is to be used. If the given directory path doesn’t already exist, and it 189 is not possible to create the given directory, or it doesn’t already exist, then testuqwordladder must print the 190 following message to stderr (with a following newline) and exit with an exit status of 14: 191
testuqwordladder: Unable to create directory "dirname" 192
where dirname is replaced by the name of the directory – as given on the command line, or ./tmp if the 193 --testdir argument is not given on the command line. If it doesn’t already exist, the directory must be 194 created with at least rwx permissions for the owner. Note that it is sufficient to check that the directory path 195 already exists; it is not necessary to check that an existing path with that name is a directory. 196
If the given test directory can be created (or already exists) then testuqwordladder must iterate over each 197 test job from the job specification file. Each test job will need three expected output files in the test directory. 198 The three files are 199
-
testid.stdout – containing the expected standard output 200
-
testid.stderr – containing the expected standard error 201
-
testid.exitstatus – containing the expected exit status (an integer printed as ASCII text, followed by 202 a newline). 203
where testid is replaced by the test ID for that test job (i.e. all the characters before the first tab character 204 on the relevant line in the job specification file).1675 205 If any of the expected output files is missing, or if any of the files have a modification time before the 206 modification time of the job specification file, or if --rebuild is specified on the testuqwordladder command 207 line, then testuqwordladder must print the following1675to stdout (with a following newline) and the expected 208 output files for that test must all be (re)generated: 209
Test testid: generating expected output 210
where testid is replaced by the test ID. 211 Regeneration of the expected output files involves running good-uqwordladder with the command line 212 arguments specified in the test job specification and with standard input coming from the input file specified 213 in the test job specification. The standard output that results from this execution must be saved to the 214 testid.stdout file in the test directory; the standard error must be saved to the testid.stderr files in the test 215 directory; and the exit status must be saved as a text value with a following newline to the testid.exitstatus 216 file. Note that good-uqwordladder must be run as a direct child of testuqwordladder (i.e. not a grand child).1675 217 As an example, if the job specification file contains a line like this (where indicates a tab character): 218
1 1.1
/dev/null
--start head
--end tail
then the equivalent of the following shell commands will be run to (re)generate the expected outputs: 219
good-uqwordladder --start head --end tail < /dev/null > testdir/1.1.stdout \ 2> testdir/1.1.stderr
echo $? > testdir/1.1.exitstatus
1 2 3
where testdir is replaced by the name of the test directory.
If it is not possible to create one of the expected output files then testuqwordladder must print the following 221 message to stderr (with a following newline) and exit with an exit status of 16:1675 222
testuqwordladder: Unable to open "filename" for writing 223
where filename is replaced by the name of the file that could not be created, in the form testdir /testid .type 224 where testdir is replaced by the name of the test directory, testid is replaced by the test ID, and type is 225 replaced by one of stdout, stderr, or exitstatus. The ability to open the .stdout file for writing must be 226 checked before the .stderr file which which must be checked before the .exitstatus file. 227
Additional files in an existing test directory (i.e. files not related to the tests in a given job specification file) 228 should be left untouched – there is no need to delete them. 229
Running Test Jobs and Reporting Results 230
If the expected outputs of all of the test jobs are able to be created (or already existed and didn’t need to be 231 recreated), then testuqwordladder should move on to running the tests. 232 For each job in the job specification file (in the order specified in that file), testuqwordladder must: 233
-
Start the job (as described below in Running a Test Job) 234
-
Allow up to 1.5 seconds for the job to finish and terminate the processes that make up that job (with 235
SIGKILL) if they are still running after 1.5 seconds 236
-
Report the result of the job (as described below in Reporting the Result of a Test Job). Note that if a 237
process is killed then it will not have exited normally and the test will be reported as failing. 238
When one job is complete, the next job can be started.1675 239 Allowing up to 1.5 seconds for the job to finish can most simply be implemented by sleeping for 1.5 seconds 240 and then sending the SIGKILL signal to all processes that make up the job. (It is likely they are already dead, 241 but testuqwordladder does not need to check this prior to attempting to send a signal.) 242 To allow tests to run quicker, testuqwordladder can implement more advanced timeout functionality. 243 Rather than sleeping for 1.5 seconds after starting each test, your program can start the next test immediately 244 after the previous one finishes, provided no test runs for more than 1.5 seconds, i.e. SIGKILL is still sent to 245 the child processes after 1.5 seconds if the test is still running. 246
Running a Test Job 247 Just before running a test job, testuqwordladder must output and flush the following to standard output 248
(followed by a newline): 249 Running test: testid 250
where testid is replaced by the test ID. 251 For each individual test job, testuqwordladder must create two pipes and three child processes as shown 252 in Figure 1. The three processes must all be direct children of testuqwordladder (i.e. not grandchildren). The 253 processes are:1675 254
-
an instance of the program being tested (as specified on the testuqwordladder command line). Standard 255 input for this process must come from the input file specified in the test job. Standard output and standard 256 error must be sent to standard input of the other two processes (see below). 257
-
an instance of cmp which compares standard outputs. This cmp process must be run with one command 258 line argument – the name of the file containing the expected standard output (which will be of the form 259 testdir/testid.stdout). The standard input of this cmp process will be the standard output of the 260 program being tested. cmp must be found in the user’s PATH – do not assume a particular location for 261 cmp. 262
-
an instance of cmp which compares standard errors. This cmp process must be run with one command 263 line argument – the name of the file containing the expected standard error (which will be of the form 264 testdir/testid.stderr). The standard input of this cmp process will be the standard error of the 265 program being tested. cmp must be found in the user’s PATH – do not assume a particular location for 266
cmp.
The standard output and standard error of the cmp processes must be redirected to /dev/null.
Reporting the Result of a Test Job 269
cmp will exit normally with an exit status of 0 if the data received on standard input matches the contents of 270 the file whose name is given on the command line, otherwise it will exit with a non-zero exit status. See the 271 cmp man page for more details. 272
If any of the programs were unable to be executed then testuqwordladder must print the following to 273 standard output (followed by a newline):1675 274
Unable to execute test testid 275
where testid is replaced by the test ID. This counts as a test failure. The message must be printed at most 276 once per test, even if multiple programs were unable to be executed. No other messages are to be output about 277 this test. (If one test job fails then all test jobs are likely to fail. A possible scenario is where one of the programs 278 is not found in the user’s PATH. Your testuqwordladder program must not assume that future executions will 279 fail – all tests must be attempted, even if the same failure occurs.) 280
If the three processes run, then for one individual test testuqwordladder must report the following – in 281 this given order. (In all of the messages below, testid is replaced by the test ID. All messages are sent to 282 testuqwordladder’s standard output and are terminated by a single newline, and must be flushed at the time 283 of printing.) 284
• If the standard outputs of the two programs match (as determined by the exit status of the relevant cmp 285 instance), then testuqwordladder must print the following: 286 Job testid : Stdout matches 287 If the standard outputs do not match, then testuqwordladder must print the following: 288 Job testid : Stdout differs 289
• If the standard errors of the two programs match (as determined by the exit status of the relevant cmp 290 instance), then testuqwordladder must print the following: 291 Job testid : Stderr matches 292 If the standard errors do not match, then testuqwordladder must print the following: 293 Job testid : Stderr differs 294
• If the program under test exits normally and the exit status of the program under test matches the 295 expected exit status (found in the relevant .exitstatus file), then testuqwordladder must print the 296
following:
Job testid : Exit status
otherwise it must print:
Job testid : Exit status
Reporting the Overall Result
297 matches 298 299 differs1675 300
A test passes if the standard output, standard error and exit statuses of the program being tested match the 302 expected outputs in the relevant files generated earlier. 303 When the running of test jobs has finished and at least one test has been completed, then testuqwordladder 304 must output the following message to stdout (followed by a newline): 305
testuqwordladder: M out of N tests passed 306
where M is replaced by the number of tests that passed, and N is replaced by the number of tests that have 307 been completed (which may be fewer than the number of the tests in the job specification file if the tests are 308 interrupted – see below). 309
If at least one test has been completed and all tests that have been run have passed then testuqwordladder 310 must exit with exit status 0 (indicating success). Otherwise, testuqwordladder must exit with exit status 11. 311
Interrupting the Tests 312
If testuqwordladder receives a SIGINT (as usually sent by pressing Ctrl-C) then it should abort any test in 313 progress (by killing and reaping the processes), not commence any more test jobs, and report the overall result 314 (as described above) based on the tests that have been completed prior to the interruption (i.e. excluding any 315 test in progress at the time of interruption). If no tests have been completed prior to the interruption then 316 testuqwordladder must output the following message to stdout (followed by a newline) and exit with exit 317
status 4:
testuqwordladder: No tests completedon 1.1Your program is permitted to use a single non-struct global variable to implement signal handling. 320
We will not test SIGINT being sent to your testuqwordladder program during the generation of expected 321 output files – which should be quite quick. We will only test SIGINT being sent after test jobs are started. 322
Other Functionality 323
testuqwordladder must free all dynamically allocated memory before exiting. (This requirement does not 324 apply to child processes of testuqwordladder.)1675 325 Child processes created by testuqwordladder must not inherit any unnecessary open file descriptors opened 326 by testuqwordladder. (File descriptors inherited from testuqwordladder’s parent must remain open.) 327 testuqwordladder is not to leave behind any orphan processes (i.e. when testuqwordladder exits then 328 none of its children must still be running). testuqwordladder is also not to leave behind any zombie processes 329 – all child processes from one test must be reaped before the next test starts. 330 testuqwordladder should not busy wait, i.e. it should not repeatedly check for something (e.g. process 331 termination) in a loop. This means that use of the WNOHANG option when waiting is not permitted. 332
Assumptions 333
We won’t run any tests that violate the assumptions below and your program can behave in any way that you 334 like (including crashing) if one of these assumptions turns out not to be true. You can assume the following: 335
-
good-uqwordladder will always be found in the user’s PATH and execution will not fail. 336
-
good-uqwordladder will always run in a reasonable time (i.e. there is no need for a timeout on 337 good-uqwordladder when (re)generating expected output files). 338
-
good-uqwordladder will always exit normally, i.e. will not exit due to receiving a signal. 339
-
fork() always succeeds (assuming your program calls fork() a reasonable number of times). 340
-
malloc() always succeeds (assuming your program is using a reasonable amount of memory).1675 341
-
pipe() always succeeds (assuming your program is using a reasonable number of file descriptors). 342
-
If a job input file or job specification file exists and is readable at some point in testuqwordladder’s 343 execution then it will continue to exist and be readable during that execution (assuming your program 344 does not change that). 345
CSSE7231 Functionality 346 In addition to the functionality above, CSSE7231 students are expected to implement additional functionality 347
as described below: 348 Setting Environment Variables in Tests (CSSE7231 only) 349
CSSE7231 students must support a variation on the job specification file format that allows environment vari- 350 ables to be specified for each test. Such environment variable definitions take the form: 351
NAME=value 352 and will optionally be found in the job specification lines after the input file name and before the command line 353
arguments: 354
test-id input-file-name[ NAME1=value1[ NAME2=value2 ...]][ arg1[ arg2 ...]] 355
Square brackets ([ ]) indicate optional fields. As before, all fields are separated by tab characters (illustrated 356
with here). Any fields immediately after the job input file name that contain an equals (‘=’) sign are to be 357 treated as environment variable values. The first field (if any) found after the job input file name that does not 358 contain an equals sign is to be treated as the first argument field (and all remaining fields are argument fields). 359
An example of such a test specification file might be:1675
1 2 3
1.1
testfiles/1.1.in
1.2
/dev/null
1.3
testfiles/1.3.in
UQWORDLADDER_WORD_1=HEAD
UQWORDLADDER_WORD_1=HEAD
--length 8
UQWORDLADDER_WORD_2=TAIL --dictionary ./words --to tail --from BAD=word
The first test in this example (1.1) specifies two environment variables and no command line arguments. 360 The second test (1.2) specifies one environment variable and four command line arguments. The third test (1.3) 361 specifies no environment variables and four command line arguments. 362
Any environment variable definitions must be applied for the running of good-uqwordladder to (re)create 363 expected output files (if necessary) and for the running of the program under test during each test job. They 364 must be applied only for the running of the specified test and not for other tests. The variable definitions 365 must not be applied for any other child processes of testuqwordladder (e.g. cmp). All environment variables 366 inherited from the parent process (e.g. PATH) must be unchanged (unless overridden by one of these definitions).1675 367
Showing Output Differences (CSSE7231 only) 368
CSSE7231 students must also implement a variation on the processes to be run within a test. This will show the 369 differences found between the expected and actual outputs, rather than just reporting that there are differences. 370 A limit can be put on the number of lines of difference shown. 371
If the --diff argument is specified on the command line (with an associated value argument) then 372 testuqwordladder must use a different set of processes when running the test jobs – as shown in Figure 2. 373 (Note that the (re)generation of expected output remains the same.) 374
Figure 2: Processes (A to E) and pipes (1 to 4) to be created by testuqwordladder for each test job when --diff is specified on the testuqwordladder command line.
Instead of using cmp, testuqwordladder must instead use diff run with two command line arguments: 375 the name of the file containing the expected output as the first argument and “-” (a single hyphen without 376 the quotes) as the second argument. This causes diff to compare the named file against its standard input. 377 The standard output of each diff process must be sent to the standard input of a process 1675running head with 378 two command line arguments: “-n” (without the quotes) followed by the --diff value argument (N ) from the 379 testuqwordladder command line. This argument should be a number, indicating the maximum number of 380 lines to output, but your program does not need to check that this argument is a number – it can just pass the 381 string from the testuqwordladder command line to each head command line. 382
The standard output of each head command must be sent to testuqwordladder’s standard output. The 383 standard error of the diff and head processes must be sent to /dev/null. 384 Determination of whether standard output and standard error of the program being tested match the 385 expected values must be based on the exit status of the respective diff processes which will be zero if they 386 match and non-zero if they do not match. 387
Example testuqwordladder Output 388
In this section we give examples of expected testuqwordladder output for some given job specification files 389 and command lines. Note that tab characters are shown by the symbol. 390 Note that these examples, like provided test-cases, are not exhaustive. You need to implement the program 391
specification as it is written, and not just code for these few examples.1675 Example One
Consider a job specification file as follows (named example1):
1 2
1 2 3 4 5 6 7 8 9
10 11
In the following runs of testuqwordladder, assume that ./uqwordladder is a fully functional program. 395 Assume that before the first run the directory ./tmp does not exist. 396
$ ./testuqwordladder example1 ./uqwordladder Test test 1: generating expected output
Running test: test 1
Job test 1: Stdout matches
Job test 1: Stderr matches
Job test 1: Exit status matches
|
testuqwordladder: 1 out of 1 tests passed
$ ./testuqwordladder example1 ./non-existent-program Running test: test 1
Unable to execute test test 1
testuqwordladder: 0 out of 1 tests passed
|
1 2
1 2 3 4 5 6 7 8 9
10 11 12
# One job, initial and target words provided, immediate EOF on stdin test 1 /dev/null --from head --to tail
Note that in the first run of testuqwordladder, because the /tmp directory does not exist, the expected 397 output for the test is generated – but this does not happen in the second run of testuqwordladder because 398 these files already exist and are not out of date. Note also that there is up to a 1.5 second delay between lines 399 3 and 4, and between lines 9 and 10. 400
Example Two 401
In the following example, assume that ./buggy-uqwordladder works correctly except that it exits with a seg- 402 mentation fault when given an empty dictionary. Assume that testfiles/2.1.in exists. The job specification 403 file example2 has the following contents:1675 404
1.1 /dev/null --from head --dictionary /dev/null --to tail 2.1 testfiles/2.1.in --length 4 --from alphabet
$ ./testuqwordladder example2 ./buggy-uqwordladder Test 1.1: generating expected output
Test 2.1: generating expected output
Running test: 1.1
Job 1.1: Stdout differs
|
Job 1.1: Stderr matches
Job 1.1: Exit status differs
Running test: 2.1
Job 2.1: Stdout matches
Job 2.1: Stderr matches
Job 2.1: Exit status matches
|
testuqwordladder: 1 out of 2 tests passed
|
Provided Library: libcsse2310a3 405
A library has been provided to you with the following functions which your program may use. See the man 406 pages on moss for more details on these library functions.1675 407 char* read_line(FILE *stream); 408 The function attempts to read a line of text from the specified stream, allocating memory for it, and returning 409 the buffer. 410 char **split_string(char* line, char delimiter); 411 This function will split a string into substrings based on a given delimiter character. 412 int compare_timespecs(struct timespec t1, struct timespec t2); 413 This function will compare two times represented in struct timespec types and determine which is earlier 414 than the other. 415
To use the library, you will need to add #include <csse2310a3.h> to your code and use the compiler flag 416 -I/local/courses/csse2310/include when compiling your code so that the compiler can find the include 417 file. You will also need to link with the library containing this function. To do this, use the compiler arguments 418 -L/local/courses/csse2310/lib -lcsse2310a3.1675 419
Style 420 Your program must follow version 2.4.1 of the CSSE2310/CSSE7231 C programming style guide available on 421
the course Blackboard site. 422 Hints 423
-
Make sure you review the lecture/contact content and supplementary videos and examples and the Ed 424 Lessons exercises from weeks 6 and 7. These provide necessary background for this assignment and there 425 may be code snippets that you can reuse or adapt. 426
-
While not mandatory, the provided library functions will make your life a lot easier – use them! 427
-
You can examine the file descriptors associated with a process by running ls -l /proc/PID/fd where 428
PID is the process ID of the process. 429
-
The system call mkdir() can be used to create a directory path – see the man page for details. (Run “man 430
2 mkdir” to see the man page.) mkdir() will fail with errno being EEXIST if the path already exists. 431
-
The system call stat() can be used to find out information about a file, e.g. its modification time can be 432 found in the st_mtim member of the stat structure – see the man page for details. This st_mtim member 433 is of type struct timespec which is defined in the nanosleep() man page. 434
-
usleep() or nanosleep() can be used to sleep for a non-integer number of seconds. 435
-
Execution (exec) failure can be detected by having the child process return an unexpected exit status to 436
the parent (e.g. 99). 437
-
Consider the use of sigtimedwait() for SIGCHLD if you’re implementing the more advanced timeout 438
functionality. 439
-
(CSSE7231 students) You may find the putenv() function useful.1675 440
-
A demo program will be provided: demo-testuqwordladder – you can check the expected behaviour of 441 your testuqwordladder by running this program. Test scripts (testa3.sh and reptesta3.sh) will also 442 be provided. You are strongly encouraged to use them. 443
Suggested Approach 444
It is suggested that you write your program using the following steps. Test your program at each stage and 445 commit to your SVN repository frequently. Note that the specification text above is the definitive description 446 of the expected program behaviour. The list below does not cover all required functionality but will get you 447 started. 448
-
Write a program to parse the expected command line arguments and handle usage errors. 449
-
Write code to read and parse the job specification file and save it to a data structure (and output error 450
messages if problems are found).1675 451
-
Write code that attempts to create the test directory 452
-
Write code to generate the expected output files (all the time). 453
-
Add code to check whether the expected output files need to be regenerated 454
-
Write code that creates the necessary pipes and correctly starts the three processes associated with one 455 test job (with the necessary command line arguments). 456
Forbidden Functions 457
You must not use any of the following C statements/directives/etc. If you do so, you will get zero (0) 1675marks 458
for the assignment. 459
• goto 460 • #pragma 461 • gcc attributes1675 462
You must not use any of the following C functions. If you do so, you will get zero (0) marks for any test case 463 that calls the function. 464
-
longjmp() and equivalent functions 465
-
system() 466
-
popen()1675 467
-
mkfifo() or mkfifoat() 468
-
signal(), sigset() – you should use sigaction() instead 469
-
waitpid() or waitid() using the WNOHANG option – these functions are permitted without that option 470
-
execlp() or execvp() or related family members to execute any program other than good-uqwordladder, 471 the program under test, cmp, diff, or head. (The last two are for CSSE7231 students.) 472
-
Functions described in the man page as non standard, e.g. execvpe(), strcasestr() 473 Submission 474
Your submission must include all source and any other required files (in particular you must submit a Makefile). 475 Do not submit compiled files (e.g. .o files and compiled programs).1675 476
Your program testuqwordladder must build on moss.labs.eait.uq.edu.au and in the Gradescope test 477 environment with the command: 478 make 479
Your program must be compiled with gcc with at least the following options: 480 -pedantic -Wall -std=gnu99 481
You are not permitted to disable warnings or use pragmas to hide them. You may not use source files other 482 than .c and .h files as part of the build process – such files will be removed before building your program. 483
If any errors result from the make command (i.e. no executable is created) then you will receive 0 marks 484 for functionality (see below). Any code without academic merit will be removed from your program before 485 compilation is attempted (and if compilation fails, you will receive 0 marks for functionality). 486
Your program must not invoke other programs or use non-standard headers/libraries other than those 487 explicity described in this specification.1675 488
Your assignment submission must be committed to your subversion repository under 489
https://source.eait.uq.edu.au/svn/csse2310-sem2-sXXXXXXX/trunk/a3 490
where sXXXXXXX is your moss/UQ login ID. Only files at this top level will be marked so do not put source 491 files in subdirectories. You may create subdirectories for other purposes (e.g. your own test files) 1675but these 492 will not be considered in marking – they will not be checked out of your repository. 493
You must ensure that all files needed to compile and use your assignment (including a Makefile) are com- 494 mitted and within the trunk/a3 directory in your repository (and not within a subdirectory or some other part 495 of your repository) and not just sitting in your working directory. Do not commit compiled files or binaries. 496 You are strongly encouraged to check out a clean copy for testing purposes.1675 497
To submit your assignment, you must run the command 498
2310createzip a3 499
on moss and then submit the resulting zip file on Blackboard (a GradeScope submission link will be made 500 available in the Assessment area on the CSSE2310/7231 Blackboard site)6. The zip file will be named 501
sXXXXXXX_csse2310_a3_timestamp.zip 502 where sXXXXXXX is replaced by your moss/UQ login ID and timestamp is replaced by a timestamp indicating 503 the time that the zip file was created. 504
The 2310createzip tool will check out the latest version of your assignment from the Subversion repository, 505 ensure it builds with the command ‘make’, and if so, will create a zip file that contains those files and your 506 Subversion commit history and a checksum of the zip file contents. You may be asked for your password as 507 part of this process in order to check out your submission from your repository. You will be asked to confirm 508 references in your code.1675 509
You must not create the zip file using some other mechanism and you must not modify the zip file prior 510 to submission. If you do so, you will receive zero marks. Your submission time will be the time that the file 511 is submitted via GradeScope on Blackboard, and not the time of your last repository commit nor the time of 512 creation of your submission zip file. 513
We will mark your last submission, even if that is after the deadline and you made submissions before the 514 deadline. Any submissions after the deadline7 will incur a late penalty – see 1675the CSSE2310/7231 course profile 515 for details. 516
Note that Gradescope will run the test suite immediately after you submit. When complete8 you will be 517 able to see the results of the “public” tests. 518
