Name:
Answer Key

SEE BODY TAG

SEE BODY TAG

SEE BODY TAG

Write programs for the following questions.
Begin each programs with a comment that includes your name, the date, and the question number, and a one-line description of what the script does.
Upload each programs to the course website.

1. #### Datafile handling

Name program script "q:file".

Download this data file:

Write a Python3 program that gets the name of this file, reads it, and processes the contents.

Each line will either be a blank line, a comment line that starts with a "#", or a data line that contains three floating point values. The first number on each data line is a date, the second number is average temperature, and the third number is monthly rainfall. The dates are floating point numbers representing a year and a month: for example, 2021.0 is January 2021, 2021.083 is February 2021, 2021.25 is March 2021, and 2021.333 is April 2021.

Temperatures are in degrees Fahrenheit and rainfall is in inches; both are weighted averages that include the previous 12 months. HOWEVER, at least one of the temperature or rainfall data points is missing, and is replaced by the string "None". You must split each line into three pieces, and convert them into floating point numbers unless the piece is "None"; if it's "None", omit it from the average.

Write your program to do the following:

1. [5 pts]

Get the filename as the first (and only) command-line argument. If there isn't a command-line argument, prompt for a data-file name, and get the name from user input. Open the file and read each line. Ignore blank lines and comments, and collect the values into three lists — one each for dates, temperatures, and rainfalls.

2. [10 pts]

Calculate and print out the average (monthly) temperature and rainfall, over all years and all months (i.e. over all data points in the file). Your output should look similar to this:

```\$ python3 q_file.py rolling-averages-12month.data
average temperature: 53.594 degrees F
average rainfall: 2.833 inches

⋮

```
3. [10 pts]

Calculate and print out the average (monthly) temperature and rainfall, for all months in the years 1980 to 2000; for all months in the years 1990 t0 2010; and for all months in the years 2000 to 2020. Your output should look similar to this:

```
⋮

average temperature 1980 - 2000: 52.523 degrees F
average rainfall 1980 - 2000: 2.892 inches

average temperature 1980 - 2000: 53.058 degrees F
average rainfall 1980 - 2000: 2.822 inches

average temperature 1980 - 2000: 54.683 degrees F
average rainfall 1980 - 2000: 2.889 inches

\$
```

You don't need the numpy module for this problem.

2. #### Website reading and regular expressions

Name this program "q:webpage".

Write a program that can be run from the command line, with one or more command-line arguments. The arguments will be URLs (for example, "http://bloomu.edu"). The script should do each of the following tasks:

1. [10 pts] Retrieve and open each URL, and read in its contents. Remember that the contents will read in as bytes; you must decode them to text.

Some possible decoders include:

• "utf-8"
• "latin-1"
• "cp1252"
• "ebcdic"
Report which decoders failed, and which one succeeded.

Report each URL and the total length of its contents (note that the URLs' contents may change from day to day).

2. [15 pts] Use a regular expression to find all all URLs in the page that you retrieved. A URL is a string that starts with `"http` (includingt the doublequote), includes more than one character that is not a doublequote, and ends with a doublequote (`"`).

Note that automatically generated webpages might not be divided into separate lines, so the "read()" method is most appropriate.

Print all the URLs found, one per line, followed by the number of links found.

3. The results should look similar to this:
```\$ python3 q_webpage.py  http://montcs.bloomu.edu/  http://whitehouse.gov  https://google.com  http://microsoft.com
used utf-8
URL http://montcs.bloomu.edu/:  6726 bytes
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"
"http://www.w3.org/1999/xhtml"
"http://time.gov/widget/NISTtimewidget.swf"
"https://bloomu.edu/computer-science/"
"http://bloomu.edu"
"http://www.bloomu.edu/sites/default/files/images/BUlogoMB.jpg"
"http://bloomu.edu/"
URL http://montcs.bloomu.edu/  7 links

used utf-8
URL http://whitehouse.gov:  118491 bytes
"https://"
"https://gmpg.org/xfn/11"
"https://www.whitehouse.gov/"
"https://www.whitehouse.gov/"
"https://www.facebook.com/WhiteHouse/"
"https://www.whitehouse.gov/wp-content/uploads/2021/01/wh_social-share-1100x740.png"
"https://schema.org"
"https://www.whitehouse.gov/#organization"

⋮

"https://mail.google.com/mail/?tab=wm"
"https://drive.google.com/?tab=wo"
"https://www.google.com/intl/en/about/products?tab=wh"
"http://www.google.com/history/optout?hl=en"
"https://accounts.google.com/ServiceLogin?hl=en&passive=true&continue=https://www.google.com/&ec=GAZAAQ"
URL https://google.com  14 links

used utf-8
URL http://microsoft.com:  316 bytes
URL http://microsoft.com  0 links

\$
```
3. #### matplotlib

Name this program "q:trends". Re-use (or download) this data file:

Write a Python3 program that gets the name of this file, reads it, and displays a graph of the contents.

Split each line into three pieces, and convert them into floating point numbers. If one of the pieces is "None", convert it to the Python value `None`.

Write your program to do the following: 1. [5 pts]

Get the filename as the first (and only) command-line argument. If there isn't a command-line argument, prompt for a data-file name, and get the name from user input. Open the file and read each line. Ignore blank lines and comments, and collect the values into three lists — one each for dates, temperatures, and rainfalls.

2. [10 pts] Produce a figure containing two graphs, like the figure shown at right above.
3. [10 pts] Produce another figure containing the same two graphs, like the second one shown below. This one must include a title, shared X axis, dotted lines of different colors, and a legend on each graph. The marker symbols are the comma, "," for a small dot. The figure must be saved to a file as 'weather.png'. You don't need the numpy module for this problem.

4. #### numpy and matplotlib

Name this script "q:numpy".

Write a program that uses numpy to create a surface z = f(x, y), and matplotlib to display it. Write it to do the following:
1. [10 pts]

Using numpy, define a function f(x, y) as follows:

• theta = (x2 + y2)0.5
• z = sin (theta) * cos (theta/2) * e-1/2
• f(x,y) = z

Create X and Y vectors that range from -1.5 to +9 and -2.5 to +6, and are each 300 points in length. (hint np.linspace) Then use `np.meshgrid` to create 2-dimensional arrays, and use them to create the surface z = f(x, y).

Print the "shape" and the mean (average) value of the surface array. Your output should look like this:

```\$ python3  q-numpy.py
surface shape: (300, 300)
surface mean: 0.0851786933301267
```
2. [10 pts]

Make a figure containing a 3-dimensional surface plot ("fig.plot_surface("), and a 2-dimensional contour plot. Give the surface plot an alpha parameter of 0.7 (`alpha=0.7`). Use the "pcolormesh()" function to make the contour plot, and add a colorbar to it.

Use the "set_xlabel()", "set_ylabel()", and "set_zlabel()" functions to add labels to the axes. Add a colormap of your choice (possibilities include "cm.hot", "cm.copper", "cm.cool", "cm.terrain", "cm.gnuplot2", and many others). Optionally, add contours to each of the plots. Your figure should look similar to figure at right (shown with "terrain" coor map, and optional contours).

3. [5 pts]

Create a reshaped copy of your surface, that has 250 rows by 360 columns.

Print the "shape" and the mean (average) value of the new array. Your output should look like this:

```\$ python3  q-numpy.py
surface2 shape: (120, 750)
surface2 mean: 0.0851786933301267
```
4. [5 pts]

Produce another figure containing a 3-diemensional surface plot and a 2-dimensional contour plot of the reshaped surface. Use a different color map than you used for the first figure.