COMP3322 Modern Technologies on World Wide Web - Assignment Four: Big Cities
COMP3322 Modern Technologies on World Wide Web
Assignment Four
Total 12 points
Deadline: 23:59 December 3, 2023
Overview
Write an express.js program and name it index.js. This program provides the API to get data about big cities from a MongoDB server.
Objectives
-
A learning activity to support ILO 1 and ILO 2.
-
To practice how to use Node, Express, MongoDB, and Mongoose to create a simple REST API.
Specification
Assume you are using the MongoDB server running on the course’s node.js docker container with the service name mongodb listening to the default port 27017.
The database is named "bigcities" and it contains a collection called "cities". The collection consists of 34800 cities with a population of at least 10000. The data for this dataset is sourced from the GeoNames geographical database (https://www.geonames.org/about.html). Each record in the collection consists of 9 fields: _id, Name, “ASCII Name”, “ISO Alpha-2”, “ISO Name EN”, Population, Timezone, “Modification date”, and Coordinates.
ISO Alpha-2
The ISO 3166
Alpha-2 country
code
AR
HK
Coordinates
The latitude and longitude values of the city -32.81636, -61.39493 22.27832, 114.17469
You will be using the provided framework for developing your program. You can download the template file (template.txt) from the course’s Moodle site.
index.js
Example 1 Example 2
_id
The id of record in Geonames database 3862981
1819729
Population
The population of the city
36000
7482500
Name
The name of the city (in UTF8)
Cañada de Gómez
Hong Kong
Timezone
The IANA timezone
ID
ASCII Name
The name of the city (in ASCII) Canada de Gomez Hong Kong
Modification date The date of last modification 2020-06-10
ISO Name EN
The English name of the Alpha-2 code Argentina
Hong Kong, China
Example 1 Example 2
America/Argentina/ Cordoba Asia/Hong Kong
2021-09-09
Download the big cities dataset (bigcities.csv) from the course’s Moodle site. Import the data to the
bigcities database for the tests.
const express = require('express')
const app = express();
/* Implement the logic here */
// error handler
app.use(function(err, req, res, next) {
res.status(err.status || 500);
res.json({'error': err.message});
});
app.listen(3000, () => {
console.log('Weather app listening on port 8000!')
});
TASK A
Use the command mongoimport to import the CSV file to the MongoDB server. Here are the steps to import the data to your docker’s mongodb server.
-
Use Windows Explorer or Mac Finder to go to the data/db folder (which is inside the Node- dev folder).
-
Copy the bigcities.csv file there.
-
Access the docker desktop and open a terminal for the c33322-mongo container.
-
In the terminal, type this command (in one line):
mongoimport -d=bigcities -c=cities --type=csv --headerline --columnsHaveTypes --file=bigcities.csv
Write the code to set up a connection to the MongoDB server using Mongoose. Use the following schema to access the database.
Schema {
Name: String,
'ASCII Name': String,
'ISO Alpha-2': String,
'ISO Name EN': String,
Population: Number,
Timezone: String,
'Modification date': String,
Coordinates: String
}
Write the code that monitors the database connection and terminates the program if the connection to the database is lost.
TASK B
Write a routing endpoint to handle all GET requests to the URL http://localhost:3000/cities/v1/all?gte=xxxxx<e=yyyyy
for retrieving the entire big cities dataset or a portion of the dataset based on the population range defined in the query string. The server should respond with a JSON message and an appropriate HTTP status code that reflects the completion status of the GET request to the client.
Situations:
1. GET /cities/v1/all
When the GET request is made without a query string,
the program retrieves the entire dataset from the
database. It then converts the Coordinates field to an
object with two properties: ‘lat’ and ‘lng’. These
properties represent the latitude and longitude values
(both of type Number) of the city. The program returns
the entire dataset in JSON format to the client with the
HTTP status code 200. The returned JSON message is an
array that contains all the documents, ordered by the _id
field.
2. GET /cities/v1/all?gte=xxxxx
GET /cites/v1/all?lte=yyyyy
GET /cities/v1/all?gte=xxxxx<e=yyyyy
When the GET request includes a query string with the ‘gte’ and/or ‘lte’ parameters, the
program retrieves the dataset from the database based on the population range specified by the
query string. ‘gte’ stands for and ‘lte’ stands for . For example, the program retrieves all cities
with a population one million for the parameter gte=1000000. Another example, the program
retrieves all cities with a population between 500000 x 1000000 for the parameters
gte=500000<e=1000000. After retrieving the dataset, the program should convert the
Coordinates field to an object and sort the dataset in descending order of population. The
program then returns the dataset in JSON format to the client with HTTP status code 200.
The program should return a JSON string '{"error":"No record for this population range"}' with the HTTP status code 404 when it could not find any documents matching the limit defined by the parameters, e.g., lte=1000>e=10000.
3. When the program experiences an error (e.g., database issue), it returns the HTTP status code 500 with a JSON string '{"error":$message}', where $message stands for the error message of that error event.
TASK C
Create a routing endpoint that handles all GET requests to the URLs http://localhost:3000/cities/v1/alpha
http://localhost:3000/cities/v1/alpha/{code}
for retrieving all the alpha codes in the dataset or all the documents in the dataset that match a specified alpha code in the URL path. The server should respond with a JSON message and the appropriate HTTP status code to indicate the completion status of the GET request.
Situations:
-
/cities/v1/alpha
With this GET request, the program searches the database to find all unique alpha-2 codes in the dataset. For each alpha-2 code, the program creates an object with two properties: 'code' and 'name', which contain the values from the ISO Alpha-2 and ISO Name EN fields, respectively. The program then groups all alpha-2 code objects into an array and sorts them in ascending order based on the alpha-2 codes. Finally, the program returns this array object as a JSON message to the client with a status code of 200. -
/cities/v1/alpha/{code}
With this GET request, the program searches the database to retrieve all documents that match the specified alpha code in the path. For example, if the requested path is '/cities/v1/alpha/HK', the program will find all documents with the 'HK' alpha-2 code. For each matched document, the program retrieves the following fields: “ASCII Name”, Population, Timezone, and Coordinates. It converts the Coordinates field to an object and groups all matched documents in descending order based on population. The program then returns this array object as a JSON message to the client with status code 200.The program should return a JSON string ‘{“error”:”No record for this alpha code”}’ with the HTTP status code 404 when it could not find any documents matching the requested alpha code.
-
When the program experiences an error (e.g., database issue), it returns the HTTP status code 500 with a JSON string ‘{“error”:$message}’, where $message stands for the error message of that error event.
TASK D
Create a routing endpoint that handles all GET requests to the URLs
http://localhost:3000/cities/v1/region http://localhost:3000/cities/v1/region/{region}
for retrieving all the regions in the dataset or all the documents in the dataset that match a specified region in the URL path. In response, the server returns a JSON message and appropriate HTTP status code to the client, which reflects the completion status of the GET request.
Situations:
-
/cities/v1/region
With this request, the program retrieves the Timezone field of all documents and extracts the first component of the Timezone field to be the region. For example, if the Timezone value is "America/Argentina/Cordoba", the program will extract the region as "America". The program then returns all unique regions in the dataset as a JSON message to the client with the HTTP status code 200. The JSON message lists all regions in alphabetical order. -
/cities/v1/region/{region}
With this GET request, the program searches the database to retrieve all documents that have the first component of the Timezone field matches the specified region in the URL path. For example, if the requested path is '/cities/v1/region/Atlantic', the program will find 72 documents. For each matched document, the program retrieves only the following fields: “ASCII Name”, “ISO Alpha-2”, “ISO Name EN”, Population, Timezone, and Coordinates. It converts the Coordinates field to an object and groups all matched documents in descending order based on population. The program then returns this array object as a JSON message to the client with status code 200.The program should return a JSON string ‘{“error”:”No record for this region”}’ with
the HTTP status code 404 when it could not find any documents matching the requested region.
-
When the program experiences an error (e.g., database issue), it returns the HTTP status code
500 with a JSON string ‘{“error”:$message}’, where $message stands for the error message of that error event.
TASK E
Create a routing endpoint that handles all GET requests to the URL http://localhost:3000/cities/v1/{city}?partial=true&alpha=xx®ion=yyyy&sort=alpha|pop
ulation
for retrieving all the documents in the dataset that match the specified city in the URL path. In response, the server returns a JSON message and appropriate HTTP status code to the client, which reflects the completion status of the GET request.
Situations:
1. /cities/v1/{city}
With this GET request, the program retrieves all documents in the database that have the “ASCII Name” field exactly matches with the specified city name in the URL path. For example, when the city name is “Logan”, the program returns only one document; whereas for the city name “Paris”, it returns 4 matched documents. For each matched document, the program retrieves the following fields only: _id, “ASCII Name”, “ISO Alpha-2”, “ISO Name EN”, Population, Timezone, and Coordinates. It converts the Coordinates field to an object and groups all matched documents in ascending order based on the _id field. The program then returns this array object as a JSON message to the client with status code 200.
-
/cities/v1/{city}?partial=true
When a query string is provided with the parameter “partial=true”, the program finds all documents where the “ASCII Name” field partially matches with the specified city name in the URL path. For example, when the city name is “Logan”, the program returns 6 matched documents that have the string “Logan” in their “ASCII Name” fields. If the parameter “partial” has a value other than “true”, the program should ignore this parameter and apply the exactly match as the searching criteria. -
/cities/v1/{city}?alpha=xx
/cities/v1/{city}?region=yyyy
When the query string contains the “alpha” parameter, the
program restricts the search to documents under this alpha
code for the exactly or partially matched of the city name
(based on the partial parameter). For example, if a search is performed on the city name "Logan" with partial=true and alpha=AU, only one matched city is found.
When the query string contains the “region” parameter, the program restricts the search to documents under this region for the exactly or partially matched of the city name. For example, when searching for the city name “Logan” with partial=true and region=America, five matched cities are located.
If both the alpha and region parameters are provided, the program should ignore the region parameter as the alpha parameter should have a higher priority. -
/cities/v1/{city}?sort=alpha|population
If the sort parameter is not included, the default order will be based on the ascending order of the _id field. If the sort parameter is included with the value “alpha”, all returned results will be sorted in ascending order of the alpha code. If the sort parameter is included with the value “population”, all returned results will be sorted in the descending order of population. Otherwise, ignore other values and use the default order. -
The program should return a JSON string ‘{“error”:”No record for this city name”}’ with the HTTP status code 404 when it could not find any documents matching the requested city name with the parameters.
-
When the program experiences an error (e.g., database issue), it returns the HTTP status code 500 with a JSON string ‘{“error”:$message}’, where $message stands for the error message of that error event.
TASK F
Write a routing endpoint to intercept all other request types and paths, which are not defined in previous tasks. Return a JSON string with the HTTP status code 400. For example, for the request POST /cities/v1/all HTTP/1.1, we get the response '{"error":"Cannot POST /cities/v1/all"}'; for the request GET /cities/alpha/AU HTTP/1.1, we get the response '{"error":"Cannot GET /cities/alpha/AU"}'.
Resources
You are provided with the following files.
-
template.txt – the framework for the index.js file.
-
bigcities.csv – the big cities data set.
Testing platform
We shall run the server program in the node-dev container set and use Curl and Firefox to test the API.
Submission
Please finish this assignment before 23:59 December 3, 2023 Sunday. Submit the following files:
1. A JSON file – use mongoexport to export the whole collection from the bigcities database.
Similar to the mongoimport command, you have to open a terminal at the data/db folder
and type the following command (in one line):
mongoexport -d=bigcities -c=cities --jsonArray --sort='{_id: 1}' --out=3035111999.json
Replace 3035111999 with your student ID and upload this JSON file.
-
The complete index.js program and other required files.
-
The package.json file of your express program.
Grading Policy
Points Criteria
|
2.0 |
Task A |
|
2.5 |
Task B specific set of data ▪ Error handling |
|
2.0 |
Task C
|
|
2.0 |
Task D
|
|
2.5 |
Task E
|
1.0 Task F
▪ Error handling of all unknown methods and paths
-4.0 Using any external libraries.
Plagiarism
Plagiarism is a very serious academic offence. Students should understand what constitutes plagiarism, the consequences of committing an offence of plagiarism, and how to avoid it. Please note that we may request you to explain to us how your program is functioning as well as we may also make use of software tools to detect software plagiarism.
