CSSE2310/CSSE7231 Computer Systems Principles and Programming - Assignment 3
CSSE2310/CSSE7231 — Semester 1, 2024
Assignment 3 (version 1)
Introduction 1
The goal of this assignment is to demonstrate your skills and ability in fundamental process management 2 and communication concepts (pipes and signals), and to further develop your C programming skills with a 3 moderately complex program. 4
You are to create a program (called uqfindexec) which allows users to run a specified command, or a 5 pipeline of commands, on all files in a given directory. (This is similar to the -exec functionality available with 6 the Linux find command.) The assignment will also test your ability to code to a particular programming style 7 guide, and to use a revision control system appropriately. 2717 8
Specification 60
The uqfindexec program will execute a specified command, or pipeline of commands, for each file found in a 61 specified directory. If the string {} appears in the command(s) then it will be replaced by the name of the file 62 being processed. For example, running 63
uqfindexec --dir /etc "wc -l {}" 64 will run the command 65 wc -l /etc/filename 66
for every file found in /etc – where filename is replaced by the name of each file in turn. (Overall, this will 67 report the number of lines in each file in /etc.) 68 Running 69
uqfindexec --showhidden "tr a-z A-Z < {} > {}.out" 70 will run the command 71 tr a-z A-Z < filename > filename.out 72
for every file found in the current directory – including hidden files whose names begin with ‘.’. (Overall, this 73 command capitalises the content of every file found in the current directory and saves that output to a new file 74 with the .out suffix added to the name.) 75
The command 76 uqfindexec --showhidden --dir /etc 'stat {} | grep "Change: 2023" | cut -d " " -f 2' 77 will run the pipeline 78 stat /etc/filename | grep "Change: 2023" | cut -d " " -f 2 79
on every file found in /etc – including hidden files. (This command reports the modification dates of all files 80 in /etc that were modified in 2023.) 81 Full details of the required behaviour are provided below.2717 82
Command Line Arguments 83
Your program (uqfindexec) is to accept command line arguments as follows: 84 ./uqfindexec [--dir dir] [--descend] [--report] [--showhidden] [--parallel] [cmd] 85
The square brackets ([]) indicate optional arguments (or pairs of arguments). The italics indicate placehold- 86 ers for user-supplied arguments. Any or all of the options can be specified (at most once each). The command, 87 if specified, must always be the last argument and can be assumed not to start with --. Option arguments can 88 be in any order. 89
-
--dir – if specified, this option argument is followed by the name of the directory whose files are to be 96 processed. If omitted, the current directory (.) is to be used. 97
-
--descend – if specified, this option argument indicates that all subdirectories of the given (or default) 98 directory are to be processed recursively after files in the given/default directory are processed. Support 99 for this functionality is only required for CSSE7231 students but the programs of all students must accept 100 the argument without error.2717 See details of required CSSE7231 functionality on page 8. 101
-
--parallel – if specified, this option argument indicates that the processing of the files must be performed 102 in parallel. (By default, processing must take place sequentially – one file at a time.) 103
-
--showhidden – if specified, this option argument means that hidden files (those whose names begin with 104 .) must be processed in addition to non-hidden files. When the --descend option is also specified, then 105 hidden subdirectories will also be processed. 106
-
--report – if specified, this option argument means that statistics are output to standard error when 107 uqfindexec finishes 108
-
cmd – if specified, the given command or command pipeline must be run for each file being processed. 109 Any instances of {} in the command argument must first be replaced by the name of the file being 110 processed. (More details are provided below about the syntax of the command or command pipeline.) If 111 this argument is not present then the default command ("echo {}") must be used. This default command 112 will just print the name of each file being processed2. 113
Prior to doing anything else, your program must check the command line arguments for validity. If the 114 program receives an invalid command line then it must print the (single line) message: 115
Usage: uqfindexec [--dir dir] [--descend] [--report] [--showhidden] [--parallel] [cmd] 116
to standard error (with a following newline), and exit with an exit status of 17. 117
Invalid command lines include (but may not be limited to) any of the following: 118
• The --dir option argument is given but it is not followed by an associated value argument. 119 • Any of the option arguments is listed more than once. 120 • An unexpected argument is present.2717 121 • Any argument is the empty string. 122
Checking whether the dir , and/or cmd arguments (if present) are themselves valid is not part of the usage 123 checking (other than checking that their values are not empty). The validity of these values is checked after 124 command line validity as described in the sections below – and in the same order as these sections are listed. 125
Directory Validity Checking 126 Your program must check that the nominated directory (the directory specified on the command line or otherwise 127
the default directory (.)) exists and is readable. If it is not, then your program must print the message: 128
uqfindexec: directory "dir" can not be accessed 129
to standard error (with a following newline) where dir is replaced by the name of the directory from the 130 command line (or . if no directory was specified on the command line3). The double quotes must be present. 131 Your program must then exit with an exit status of 2. 132
Command Checking 133
If the cmd argument is specified then your program must check that it is valid. A library function has been 134 provided to parse the string. You can use the return value from this function to determine whether the command 135 or command pipeline is valid. See details of this provided library function on page 9. 136
If the cmd argument is not valid, then your program must print the message:2717 137 uqfindexec: command is invalid 138 to standard error (with a following newline), and exit with an exit status of 11. 139
2Note that the default command must be implemented by forking and executing the echo command found in the PATH. Your program must not just print the file name.
3The current directory will always exist (otherwise it couldn’t be the current directory) but it may not be readable, i.e., you may not have permission to read the names of the files in the current directory even if you have permission to change into it.
Command Description 140
The cmd argument is a string that may contain a single command to be executed (possibly with additional 141 arguments) or a pipeline of such commands to be executed. The format is a simplified form of a shell command 142 with elements separated by whitespace characters. Elements in the command can be enclosed in double quotes 143 to escape special characters such as spaces, |, < and >. It is not possible to escape double quote characters or 144 the {} placeholder. 145
A single command will have a format like the following (where square brackets [ ] indicate optional elements, 146 italics indicate text to to be replaced with an appropriate argument, and an ellipsis (...) indicates that the 147 previous element is repeatable, e.g. that multiple arguments can be given): 148
cmd [arg ...] [ < inputfile ] [ > outputfile ] 149 If a standard input redirection is not specified then the command’s standard input must be inherited from 150 the command’s parent (uqfindexec). If a standard output redirection is not specified then the command’s 151 standard output must be inherited from the command’s parent (uqfindexec). The redirections can be in either 152 order. 153 A pipeline of commands will have two or more commands separated by the | symbol. Only the first command 154 may have a standard input redirection. (If not present, the standard input of the first command in the pipeline 155 will be inherited from the command’s parent - uqfindexec.) Only the last command may have a standard 156 output redirection. (If not present, the standard output of the last command in the pipeline will be inherited 157 from the command’s parent - uqfindexec.) A pipeline of commands will have the following format (using the 158 notation above): 159 cmd [arg ...] [ < inputfile] [ | cmd [arg ...] ]... | cmd [arg ...] [ > outputfile] 160 As mentioned above, a library function is provided (see Provided Library on page 9) that will parse the 161 command string so that you don’t have to write code to do this. (You will need to implement the filename 162 placeholder ({}) substitution as described under Filename Substitution below.) 163 The standard error for all commands executed will always be inherited from the parent (uqfindexec). 164
Program Operation 165
If the given directory and command are valid then your program must iterate over all files in that directory 166 and execute the command pipeline for that file. (The term command pipeline includes the case where just one 167 command is to be executed.) If the filename placeholder {} is present anywhere in the command pipeline then 168 it must be replaced by the name of the file being processed. See Filename Substitution below. 169
Only directory entries that are regular files or symbolic links to regular files are to be processed. Other entry 170 types such as those for subdirectories are to be ignored. 171 Files must be processed in the same order that the command ls uses when listing filenames. ls sorts names 172 using the strcoll() comparison function. This compares strings based on the current locale. You must use 173 the “en_AU” collation locale for comparison purposes. Do this by calling 174 setlocale(LC_COLLATE, "en_AU"); 175 somewhere in your program prior to doing any sorting. See the Hints for a function that will return directory 176 entries in the correct order. 177 By default, hidden files (i.e. those whose names begin with .) must be skipped. However, if the --showhidden 178 argument is specified on the command line then these files must also be processed. Files must be processed in 179 the same order that ls -a uses when listing filenames. Again, only regular files or symbolic links to regular 180 files are to be processed. 181
Filename Substitution 182
2717Any occurrence of the placeholder {} in the supplied command string must be replaced by the name of the file 183 being processed prior to the command being executed. The placeholder may be present in (or may be the whole 184 of) the name of an executable, an argument to an executable, the name of the file to be the standard input for 185 the first command in the pipeline, or the name of the file to be the standard output for the last command in 186 the pipeline. Multiple placeholders may be present in a command string. 187
If the --dir option is not specified on the command line then the {} placeholder must be substituted by 188 the name of the file without any path component present. 189 If the --dir option is specified on the command line then the path to the file must be included in the 190 substitution. A single slash (/) will be added between the path and the filename if the path does not have a 191
trailing /. For example:
-
if uqfindexec is run with the arguments --dir /etc then the placeholder {} will be substituted by 193 /etc/filename for each filename in /etc; 194
-
if uqfindexec is run with the arguments --dir /etc/ then the placeholder {} will be substituted by 195 /etc/filename for each filename in /etc (i.e. there is no additional / between the path and the filename); 196
-
if uqfindexec is run with the arguments --dir ./././/////./// then the placeholder {} will be sub- 197 stituted by ./././/////.///filename for each filename in the current directory. 198
Command Execution 199 2717For each file being processed, commands in the pipeline must be executed as follows. (The term pipeline here 200
includes the case where there is only one command.) If there are N files then this sequence is repeated N times. 201
-
If an input file is specified for the first (or only) command in the pipeline then it must be opened for 202 reading and if this fails then the command execution process for this file is aborted (none of the steps 203 below are undertaken for this file) and the following message must be printed to standard error (with a 204 terminating newline): 205
uqfindexec: unable to open "filename1" for reading while processing "filename2" 206
where filename1 is replaced by the name of the file that could not be opened4 and filename2 is replaced 207 by the name of the file being processed (using the same format as would result from {} placeholder 208 substitution). 209
-
If an output file is specified for the last (or only) command in the pipeline then it must be opened for 210 writing (creating the file if it does not exist, truncating the file if it does exist)5. If the open fails then the 211 command execution process for this file is aborted and the following message must be printed to standard 212 error (with a terminating newline): 213
uqfindexec: unable to open "filename1" for writing while processing "filename2" 214
where filename1 is replaced by the name of the file that could not be opened4 and filename2 is replaced 215 by the name of the file being processed (using the same format as would result from {} placeholder 216 substitution). Output files must be created with at least read and write permission for the owner (user) 217 and no permissions for others (i.e. rw????--- permissions where the ? bits can be set or not). 218
-
The commands that make up the pipeline must be executed (after appropriate creation of pipes and 219 redirection of standard input/output as required). Each command will be executed in its own child 220 process – where each process is an immediate child of uqfindexec. Executables must be searched for 221 in the user’s PATH. (Note that all processes in the pipeline must be created before any processes in the 222 pipelined are reaped.) 223
-
Parallel Mode 238
2717If --parallel is specified on the uqfindexec command line then the command execution steps above are 239 performed in a different order. All commands must be executed (or attempted to be executed) for each file 240 prior to any child processes being reaped. In other words, steps 1 to 3 above are performed for all files in the 241 directory, and then step 4 is performed for the same files – i.e. for each file in turn (in the same order), the 242 child process(es) created must be reaped in turn (from the first in the pipeline to the last). 243
Your program does not have to deal with the possibility of fork() failing due to creating too many processes. 244 Directories with “reasonable” numbers of files will be used in testing. 245 This functionality is considered to be more advanced as you will need to create data structures to record 246 many more process IDs to enable delayed reaping. 247
Interrupting the Jobs 248
2717If uqfindexec receives a SIGINT (as usually sent by pressing Ctrl-C) when running in sequential mode then it 249 should allow the current file processing job to finish (and reap any associated child processes) and not commence 250 processing any further files. If uqfindexec is running in parallel mode (i.e. --parallel was specified on the 251 command line) then the SIGINT should be ignored, unless you are a CSSE7231 student implementing the 252 --descend functionality described below. 253
Your program is permitted to use a single bool global variable to implement signal handling. 254
Note that pressing Ctrl-C on a terminal will send a signal to a whole process group – which will include the 255 children of uqfindexec. During testing for this functionality we will send a SIGINT only to uqfindexec 256
Other Requirements 257 2717Your program must must free all dynamically allocated memory before exiting.2717 (This requirement does not 258
apply to child processes of uqfindexec.)
Child processes of uqfindexec must not inherit any unnecessary open file descriptors opened by uqfindexec. 260
(Open file descriptors that uqfindexec inherits from its parent and that are passed to a child must remain open 261 in the child.) 262 uqfindexec is not to leave behind any orphan processes (i.e. when uqfindexec exits normally then none 263 of its children must still be running). uqfindexec is also not to leave behind any zombie processes – when 264 processing files sequentially, all child processes from processing one file must be reaped before commands are 265 run for the next file. 266 uqfindexec must not busy wait, i.e. it should not repeatedly check for something (e.g. process termination) 267 in a loop. This means that use of the WNOHANG option when waiting is not permitted. 268 All commands run by uqfindexec when processing files must be direct children of uqfindexec, i.e. the use 269 of grandchild processes is not permitted. 270
Exiting (Statistics output) 271
2717When uqfindexec has finished processing all the files (or has been interrupted and will not be processing further 272 files), then, if --report is specified on the command line, uqfindexec must print the following to its standard 273 error: 274
Attempted to operate on N1 files 275 - operations finished successfully for N2 files 276 - processing may have failed for N3 files 277 - processing was terminated by signal for N4 files 278 - operations unable to be executed for N5 files 279
280 where 281
• N1 is replaced by the number of files that were processed; 282 • N2 is replaced by the number of files for which every command in the pipeline exited normally with status 283
0; 284 • N3 is replaced by the number of files for which every command in the pipeline exited normally but one 285
or more of them exited with a non-zero exit status; 286 • N4 is replaced by the number of files for which some command in the pipeline exited due to being signalled; 287
• N5 is replaced by the number of files for which the pipeline was not executed (due to the input or output 289 file not being able to be opened) or for which any command in the pipeline was not able to be executed 290 (e.g. the command was not in the user’s PATH). 291
Note that N1 = N2 + N3 + N4 + N5. 292 If --report is not specified on the command line then nothing is output on exit. 293 Whether the statistics are printed or not, your program must exit with exit status: 294
• 13if any processing failed (i.e. N5 > 0) 295 • 6if no processing failed but the program was interrupted by SIGINT prior to completion of the processing 296 • 0 otherwise 297
CSSE7231 Functionality – Directory Recursion 298 2717 CSSE2310 students are not expected to implement this functionality. No marks will be awarded if you do so. 299
If the --descend argument is given on the uqfindexec command line then your program must process files 300 in subdirectories of the given (or default) directory after processing the files in that given (or default) directory. 301 After files in the given (or default) directory are processed as described above then your program must 302 iterate over all the subdirectories in that directory. Subdirectories must be processed in the same order as ls 303 lists their names. If the --showhidden argument is also given, then hidden subdirectories (those whose names 304 begin with .) must also be included – and subdirectories will be processed in the same order as ls -a lists 305 their names. (Note that . and .. are not subdirectories.) 306 After a subdirectory’s files are processed then your program must recursively descend into its subdirectories 307 before returning to the next subdirectory of the original directory. In other words, your program must undertake 308 a depth-first traversal of the directory hierarchy. For example, if the directory to be processed is /A and the 309 following directories also exist: /A/B1, /A/B2, /A/B3, /A/B1/C1, /A/B1/C2, /A/B3/D1, then the directories must 310 be processed in the following order: 311
• /A 312 • /A/B1 313 • /A/B1/C1 314 • /A/B1/C2 315 • /A/B2 316 • /A/B3 317 • /A/B3/D1 318
Note that symbolic links to directories are not considered to be subdirectories and must not be traversed. 319
Filename placeholder substitution will take place as described in Filename Substitution above with the 320 addition of subdirectory information between the given/default path and the filename. Added path elements 321 must be separated by a single /. 322
For example: 323
• if uqfindexec was run without the --dir argument and is currently processing the abc/def subdirectory 324 of the current directory, then the placeholder {} will be substituted by abc/def/filename for each 325 filename in that subdirectory 326
• if uqfindexec was run with the --descend--dir /etc arguments and is currently processing the /etc/ssh 327 directory, then the placeholder {} will be substituted by /etc/ssh/filename for each filename in that 328 subdirectory 329
If a subdirectory is inaccessible (i.e. can not be opened for reading) then your program must print the 330 following message to stderr (terminated with a newline): 331
uqfindexec: unable to access subdirectory "subdir" 332
where subdir is replaced by the name of the subdirectory (formatted as if it were a filename after {} placeholder 333 substitution). For example, if uqfindexec was run with the --descend--dir /etc arguments and it fails to 334 access the /nftables subdirectory of /etc, then the subdir name printed will be /etc/nftables. 335
If the --parallel argument is given in addition to the --descend argument then the files within a 336 (sub)directory must be processed in parallel but each (sub)directory must be processed in turn in the or- 337 der specified above. If a SIGINT signal is received while in parallel mode, then processing of all files in that 338 (sub)directory must be completed but no further subdirectories are to be processed. 339
