IERG 4210 / IEMS 5718 - Web Programming and (Internet) Security

Form I - Client-side Implementations

Review Questions

Questions

Which of the following lines(s) are incorrect, and why? (guess and try validator)

<a ref="https://www.cuhk.edu.hk/">CUHK</a>
<img src="../ierg/cuhk.png" />
<link href="styles.css" rel="stylesheet" type="text/css" /> //Place in head or body?
<li><a href="#b">About Us<a></li>

What will be the color for "Hello" and "World"?


    *{ color: red; }
    #a > #c{ color: black; }
    p.c{ color: blue; }
    p#c{ color: green; }


    Hello
    World

Fix this (about closure and more)

Click 
...
for (var i=0; i<4; i++){ 
        const item = document.createElement("li")
        item.innerHTML = "Product" + i
        item.onclick = () => {alert(i)} //Suppose to alert(1) for Product 1
        document.getElementById("list").appendChild(item)
}

Agenda

HTTP
- Introduction & Client-Server Model
- HTTP Request and Response
HTML Forms: Basics and Input Controls
Client-side Restrictions
- HTML: The use of form elements
- HTML: HTML5 Validations
- JS: Javascript Validations
Form Submission Approaches
- Traditional Form Submission
- Programmatic Form Submission
- AJAX Form Submission

Introduction to HTTP

HTTP is a text-based application-layer protocol that defines how content is requested from a client application and served by a web server.
- Work on top of IP (IPv4/IPv6) and TCP (before HTTP/3)
- Latest standard is HTTP/3, defined in RFC 9114 (HTTP/3 over UDP)
- Specifications of HTTP Request and Response Headers
Client-Server Model
- Popular servers: Apache, Nginx, Node.js, IIS, Google AppEngine
- Popular clients/agents: Chrome, Firefox, IE Edge, Safari
- (Demo) Using telnet/netcat to make a simple (text-based) request

Client-Server Model

Interaction between Browser and Web Server

Surfing the Web using netcat

        $ nc www.ie.cuhk.edu.hk 80 (assume we send the HTTP request)
        GET / HTTP/1.1
        Host: www.ie.cuhk.edu.hk
        
        HTTP/1.1 301 Moved Permanently
        Date: Wed, 03 Feb 2021 18:11:26 GMT
        Server: Apache
        Location: https://www.ie.cuhk.edu.hk/
        Content-Length: 235
        Content-Type: text/html; charset=iso-8859-1
        
        <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
        <html><head>
        <title>301 Moved Permanently</title>
        </head><body>
        <h1>Moved Permanently</h1>
        <p>The document has moved <a 
        href="https://www.ie.cuhk.edu.hk/">here</a>.</p>
        </body></html>

Typical HTTP Requests

GET request::

GET /testing?q=abc HTTP/1.1
Host: course.ie.cuhk.edu.hk

POST request:

POST /testing?q=abc HTTP/1.1
Host: course.ie.cuhk.edu.hk
Content-Length: 105
Content-Type: application/x-www-form-urlencoded

name=World&gender=M&email=xxx%40ie.cuhk.edu.hk&address=SHB%2C+CUHK%2C+NT&region=NT&action=updInfo

Specifications:
- Version: HTTP/1.0, HTTP/1.1, HTTP/2.0
- Method: GET, POST, PUT, HEAD, DELETE, TRACE, OPTIONS, CONNECT, etc...
- Parameters: query string vs. body
- Headers: hostname, content-length, content-type

Typical HTTP Response

HTTP/1.1 200 OK
Date: Mon, 26 Jan 2015 17:00:28 GMT
Content-Length: 413
Content-Type: text/html

<HTML>...</HTML>

Specifications:
- Version: HTTP/1.0, HTTP/1.1, HTTP/2.0
- Status: 1xx for Informational, 2xx for Successful, 3xx for Redirection, 4xx for Client Error, and 5xx for Server Error
- Headers: content-length, content-type, and many more for authentication, cookies, security, caching, redirection
- Body: the content

HTML Forms

The prevalent approach to solicit information from users

A <form> tag that comprises different form controls, e.g., <input>, <select>, see a typical example below:

<fieldset>
    <legend>Personal Information</legend>
    <form method="POST" action="incl/process.php"><ul><li>
    <label>Name*</label>
    <div><input type="text" name="name" required /></div>
    </li><li>
    <label>Gender*</label>
    <div><input required type="radio" name="gender" value="M" /> M 
    <input type="radio" name="gender" value="F" /> F</div>
    </li><li>
    <label>Email*</label>
    <div><input type="email" name="email" required placeholder="john@example.com" /></div>
    </li><li>
    <label>Address*</label>
    <div><textarea name="address" required></textarea></div>
    </li><li>
    <label>Region*</label>
    <div><select required name="region">
<option value="">Choose</option>
<option value="HK">HK</option>
<option value="KL">KL</option>
<option value="NT">NT</select>
</div>
    </li>
    </ul>
    <input type="hidden" name="action" value="updInfo" />
    <input type="submit" value="Send" /><span> * denotes a required field.</span>
    </form>
</fieldset>

<form> Attributes

A typical <form> takes at least two attributes:

<form method="POST" action="process.php">
    <!-- included here are some form controls -->
</form>

method="POST" or method="GET" (default: GET)
- POST is used if a request is going to incur permanent change on server data; while GET is used for retrieving data
action="process.php" (default: the current URL)
- the value takes a URL that will accept the form request
onsubmit="return false" is optional
- onsubmit is triggered when the submit button is clicked.
- return false; prevents default behaviour, stop propagation (bubble-up), and callbacks
- Often used when the form is submitted over AJAX (cancel the form submit event)
  (to be discussed in later slides)
enctype="multipart/form-data" is optional
- Default encoding is application/x-www-form-urlencodedd
- When <input type="file"/> is used for file upload

Form Controls (1/4: Most Common Controls)

A typical form control is defined as follows:

<!-- <label> is to focus on a field when clicked -->
<label for="field1">Field 1: </label>
<input type="text" name="param1" id="field1" />

Text field

First Name: <input type="text" name="firstname" value="World"/>

Password field (MUST use POST method)

Password: <input type="password" name="name" value="abc"/>

Hidden Field:

Hidden? <input type="hidden" name="action" value="updateData"/>

Form Controls (2/4: Offering Choices)

Radio box (limit to a single choice for radios under the same name)

<input type="radio" name="sex" value="M" checked="true" /> Male 
<input type="radio" name="sex" value="F" /> Female

Checkboxes (multiple choices)

<input type="checkbox" name="item[]" value="A" checked="true"/> A 
<input type="checkbox" name="item[]" value="B" /> B

Dropdown menu
(single selection; or try the multiple attribute)

Which OS do you like: 
<select name="OS">
    <option name="1">iOS</option>
    <option name="2" selected="true">Android</option>
</select>

Form Controls (3/4: More Controls)

Textarea (Multi-line text field)

Description: 
<textarea name="desc" > text to be displayed </textarea>

File Field

Photos: <input type="file" name="pics" />

Submit Button

<input type="submit" value="Go" />

Submit Image Button (Image Credit: HSBC)

<input type="image" src="pics/go.gif" />

Form Controls (4/4: HTML 5 New Controls)

Email and Date Field

<form>Email:* 
    <input type="email" name="email" required /> 
    <input type="date" /> 
</form>

URL Field with optional use of styling by new CSS selectors

<style>:valid{border:1px solid #0F0} 
    :invalid{border:1px solid #F00}</style>
<form>URL: <input type="URL" name="url" /></form>

Search Field (vs. text type)

<form><input type="search" name="q" placeholder="Search..." /></form>

Custom Pattern Field with regular expressions

<form>Amount: $
    <input type="text" name="amount" pattern="^[\d,.]+$" />
</form>

In a nutshell, HTML5 Forms introduced
- Tags with more semantic information: Built-in support of client-side validations
- New CSS pseudo-class: :valid, :invalid, :required and :optional
  (keyword for a selected element dependent on its content or external factors)
- For compatibility, check with https://caniuse.com/

Regular Expressions

A language to recognize string patterns
Refer to a Cheatsheet for reference What you must know:
- ^ - start of string; $ - end of string (IMPORTANT to validations!)
- + - one or more times; ? - 0 or 1 times; * - 0 or more times
Examples:
- Float (\d includes digits only, the char inside "[]" defines a character set):
```
^[\d\.]+$
```
- Alphanumeric (\w includes letters, digits, underscore):
```
^[\w\-, ]+$
```
- Email (apparently '\' is the escape character here ["{}" stats the no. of pattern to match]):
```
^[\w\-\/][\w\'\-\/\.]*@[\w\-]+(\.[\w\-]+)*(\.[\w]{2,6})$
```
  The regular expression for email address is readily available on Web.
  (You need to first know what constitutes a valid email address to write this regex.)
  IMPORTANT: Consult credible websites for reusable rigorous patterns!!

https://regexr.com

Short Regular Expression Test

Which of the following string(s) match the regex ^[\d\.]+$
- 42a.4
- 4.3
- 192.168.0.1
- How about this regex [+-]?([0-9]*[\.])?[0-9]+
Write a regex to check for a 10 digit number that start with 1 (e.g., SID).
Extra:
- wildcard character (dot ".")
- [^abcd]: Not character a,b,c,d; [\W] NON-alphanumeric
- (123|456) match 123 or 456; bracket "creates" a matching group ("string")

Client-side Restrictions

To inform the users early on for input errors
- To create a more interactive and responsive UI experience
- Otherwise, input errors are prompted only after form submissions (round-trip delay)
To imply a specific pattern that a user is expected to follow
- To help users enter/choose the valid data that we need
- Yet, these restrictions can be bypassed by Parameter Tampering Attacks!! Don't count on them for security!!
- Reason: A user has full control of any client-side code downloaded to his browser using Developer Tools (and other extensions: ChroPath, TamperMonkey)
Hence, you need input validations implemented on BOTH
- server-side for security enforcement, and
- client-side for better user experience.

3 Approaches of Client-side Restrictions

The use of different form controls
- e.g., Radioboxes for genders implies a restriction on only two possible values, i.e., M or F
- e.g., Dropdown menu implies a restriction on accepting some default choices
Validations with HTML5 (shown in the previous slide)
- The first built-in support of client-side validations by IE 10+, Firefox 4+, Chrome, etc.
- e.g., Email, URL, Search, and Custom fields we just see
Validations with Javascript (to be discussed in next slide)
- The programmatic way to customize input patterns
- Well-supported across browsers

Form Validations with Javascript (1/4)

Strategy: Write your code in HTML5 for new browsers; Fallback to Javascript for legacy ones

Given a form that has an HTML5 Email field,

<form id="loginForm" method="POST">
Email: <input type="email" name="em" /><br/>
Password: <input type="password" name="pw" /><br/>
<input type="submit" value="Login" />
</form>

Note: Unsupported type will fallback to an ordinary textfield

Add the title, HTML5 requried and pattern attributes

<form id="loginForm" method="POST">
    Email: <input type="email" name="em" title="valid email" required
        pattern="^[\w\-\/][\w\'\-\/\.]*@[\w\-]+(\.[\w\-]+)*(\.[\w]{2,6})$" /><br/>
    Password: <input type="password" name="pw" 
        title="valid password" required /><br/>
    <input type="submit" value="Login" />
</form>

Note:

Unsupported attributes will be ignored in legacy browsers
The regular expression for email address is readily available on Web.

Form Validations with Javascript (2/4)

To validate the form right before form submission:

<form id="loginForm" method="POST">
    Email: <input type="email" name="em" title="valid email" required pattern="^[\w\-\/][\w\'\-\/\.]*@[\w\-]+(\.[\w\-]+)*(\.[\w]{2,6})$" /><br/>
    Password: <input type="password" name="pw" title="valid password" required /><br/>
    <input type="submit" value="Login" />
</form>
<script type="text/javascript">
var loginForm = document.getElementById('loginForm');
// Do this only if the HTML5 Form Validation is absent
if (!loginForm.checkValidity || loginForm.noValidate)
    // to listen on the submit event of "loginForm"
    loginForm.onsubmit = function(){
    // a private function for displayError
    function displayErr(el,msg){alert('FieldError: ' + msg);el.focus();return false}
    // looping over the array of elements contained in the form
    for (var i = 0, p, el, els = this.elements; el = els[i]; i++) {
        // validate empty field if required attribute is present
        if (el.hasAttribute('required') && el.value == '')
        return displayErr(el, el.title + ' is required');
        // validate pattern if pattern attribute is present
        if ((p = el.getAttribute('pattern')) && !new RegExp(p).test(el.value))
        return displayErr(el, 'in' + el.title);
    }
    // If false is returned above, the form submission will be canceled;
    // If false is NOT returned, the form will submit accordingly
    }
</script>

Form Validations with Javascript (3/4)

With an HTML5-compilant browser, JS validation is ignored:

Note: POST Parameters can be accessed only by server but not JS. Hence, nothing is shown here after submission. Firebug can show what was already sent.

HTML5 Validation can be disabled with novalidate attribute:

Note: Need to perform compatibility tests? try some online services or install those in Virtual Machines

Form Validations with Javascript (4/4)

Recall the best practice: Graceful Degradation (in Lecture 2)
- if (HTML5 supported) use the native HTML5 Validation
- else if (JS supported) use the JS validation code
- else the form still works without any validations

Extend the code to also validate radio and checkbox

for (var i = 0, p, el, els = this.elements; el = els[i]; i++) {
    // validate empty field, radio and checkboxes
    if (el.hasAttribute('required')) {
    if (el.type == 'radio') {
        if (lastEl && lastEl == el.name) continue;
        for (var j = 0, chk = false, lastEl = el.name, choices = this[lastEl],
            choice; choice = choices[j]; j++)
        if (choice.checked) {chk = true; break;}
        if (!chk) return displayErr(el, 'choose a ' + el.title);
        continue;
    } else if ((el.type == 'checkbox' && !el.checked) || el.value == '') 
        return displayErr(el, el.title + ' is required');
    }
    if ((p = el.getAttribute('pattern')) && !new RegExp(p).test(el.value))
    return displayErr(el, 'in' + el.title);
}

Code Demo. Reading: MDN Client-side form validation

3 Form Submission Approaches

Traditional Form Submission (demonstrated in the previous slide)
- Triggered by a submit button or the Enter key
- Fires the submit event, where one can validate before a form submission
Programmatic Form Submission
- Recommended to use this only when submitting a form automatically
```
<form method="POST" id="buildAutoPostReq"></form>
<script type="text/javascript">document.forms[0].submit();</script>
```
- Unfortunately, programmers (incl. HSBC) who don't know <input type="image"> like to do this for images: When an image is clicked, Form.submit() will be finally called if a form is properly validated
- BAD: NO submit event is fired. Without code analysis, difficult to know whether a submission has actually occurred
AJAX Form Submission (to be discussed in the next slide)
- AJAX: Asynchronous Javascript and XML; It's all about the XMLHttpRequest API, study it before using it to submit form data

AJAX Form Submission (1/3)

Demonstration:

Advantages:
- Modern user experience
  - Eliminate page-load effect (no blank screen);
  - Only load the changed part when it "arrives"
- Using the well-known XMLHttpRequest API
  - Sends requests at background; not limited to only send form data :)
- Cancel the default form submissions
  - returns false in the submit event

AJAX: Synchronous vs. Asynchronous (1/3)

classic synchronous mode of web application

As opposed to asynchronous calls, synchronous calls are blocking (hangs) until the server returns, i.e., less efficient.

Image Source: John Resig, Pro Javascript Techniques, p.26, 2007

AJAX: Synchronous vs. Asynchronous (2/3)

Ajax asynchronous mode of web application