Skip to content
Search

Hypertext Transfer Protocol

lastmod 2025-10-02created 2014-01-0773 edits

HTTP (HyperText Transfer Protocol; North Korean standard language (munhwaeo): chobonmun jeonsong gyuyak (초본문전송규약), haipeo bonmun jeonsong gyuyak (하이퍼본문전송규약)) is a protocol for exchanging information on the WWW. It is mainly used to exchange HTML documents. It uses TCP and UDP, and uses port 80. Version 1.0 was published in 1996 and 1.1 in 1999, and 1.1 is currently the most widely used version.

HTTP is a request/response protocol between a client and a server. For example, when a web browser, the client, requests a web page or image from a server via HTTP, the server responds to the request and delivers the needed information to the user. This information is then presented to the user through an output device such as a monitor.

Data delivered over HTTP can be retrieved with a URL (Internet address) starting with http:.

Category

Libraries

Headers

  • Caching
    • Age - The time in seconds the object has been in a proxy cache.
    • Cache-Control - Directives that specify caching mechanisms in both requests and responses.
    • Clear-Site-Data - Clears browsing data (e.g. cookies, storage, cache) associated with the requesting website.
    • Expires - The date/time after which the response is considered expired.
    • Pragma - An implementation-specific header that may have various effects anywhere along the request-response chain. Used for backward compatibility with HTTP/1.0 caches where the Cache-Control header is not present.
    • Warning - A general warning field containing information about possible problems.
  • Accept-Language
  • Referer

Debugging/Tools

  • Wireshark
  • HTTPie - The ultimate aim of HTTPie is to make CLI interaction with web services user interface friendly. This tool provides an HTTP command for sending requests, all with the use of a natural syntax.
  • Fiddler is a cross-platform web debugging proxy. It can help manipulate web sessions, inspect HTTPS traffic, and monitor traffic between your computer and the Internet.
  • RESTer is available as an extension for Google Chrome and Firefox.
  • Paw is a full-featured HTTP client, which allows you to send all kinds of HTTP requests. With Paw, you can test your APIs and also explore new ones.
  • Requester is a robust client that combines the features of HTTPie, Postman, and Paw. It works only with Sublime Text editor.
  • Postman is unarguably one of the more popular HTTP clients used for debugging web applications. It allows for faster and easier API development.
  • cURL is a command-line tool that is used for sending data using URL syntax.
  • Charles Proxy - As the name says, Charles proxy is an HTTP and reverse proxy. It works by routing local traffic through it.
  • Whistle is a NodeJS-based cross-platform debugging tool to capture and manipulate HTTP, HTTPS, WebSocket, and TCP traffic.
  • mitmproxy is a popular open-source HTTPS proxy among security researchers. Use it as a CLI, web, or Python API.
  • Proxyman - Similar to the above-mentioned Paw, Proxyman is a premium native macOS web debugging proxy application.
  • HTTP Toolkit is an open-source HTTP debugger. It’s a desktop app available for Windows, Mac & Linux.
  • Hetty - An open-source HTTP toolkit for security research
  • Burp Suite - Suite is a comprehensive web application security testing platform developed by PortSwigger.
  • Requestly is a versatile web and desktop proxy tool that allows users to modify and control HTTP/HTTPS requests and responses. It offers a range of features designed to enhance web development, testing, and debugging processes.
  • vim-rest-console - Working with .rest files in vim.
  • ngrok - A tool that lets you securely access web applications in a local development environment over the Internet.
  • httpjail - A process-level network isolation tool that filters HTTP(S) requests

Basic rules

Used throughout this specification to describe the basic parsing structure. The US-ASCII coded character set is defined by ANSI X3.4-1986.

Methods

List of HTTP Method usages

  • GET - Requests the entire resource
  • HEAD - Requests resource without the body
  • POST - Adds content to an existing web resource
  • PUT - Modifies an existing web resource
  • DELETE - Removes a specified resource
  • TRACE - Shows the changes to a web resource
  • OPTIONS - Shows available HTTP methods for the URL
  • CONNECT - Converts the request connection to a transparent TCP/IP tunnel
  • PATCH - Partially modifies a web resource

Do Not Track

The Do Not Track (DO-NOT-TRACK) header is as follows. 

DNT: 1

MIME

multipart/form-data

HTTP file upload

Let's take a look at what happens when you select a file and submit your form (I've truncated the headers for brevity): 

POST /upload?upload_progress_id=12344 HTTP/1.1
Host: localhost:3000
Content-Length: 1325
Origin: http://localhost:3000
... other headers ...
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryePkpFF7tjBAqx29L

------WebKitFormBoundaryePkpFF7tjBAqx29L
Content-Disposition: form-data; name="MAX_FILE_SIZE"

100000
------WebKitFormBoundaryePkpFF7tjBAqx29L
Content-Disposition: form-data; name="uploadedfile"; filename="hello.o"
Content-Type: application/x-object

... contents of file goes here ...
------WebKitFormBoundaryePkpFF7tjBAqx29L--

Caution

-- is prepended to the boundary that separates the contents, and the last boundary also has -- appended at the end.

How to reliably send an HTTP request as a user leaves a page

Sometimes you want to log something when a user navigates to another page or submits a form.

Putting fetch in a click event does not guarantee that it will be sent

So you try doing the Fetch and then navigating via window.location, but that is not guaranteed either (because XHR is asynchronous & non-blocking)

Handling fetch with Promise await works, but it has drawbacks:

  • Processing becomes slower, which hurts the user experience
  • Not as reliable as expected. Cases like closing the browser tab cannot be handled

There are several options for having the browser preserve the HTTP request:

  • With the keepalive flag on Fetch, the request is kept even if the page is terminated
  • Simpler functions such as Navigator.sendBeacon()
  • Browsers also support the ping attribute on the a tag (drawbacks: only usable on links, not supported by default in FF)

So which should you use?

  • fetch() + keepalive when possible
    • Supported by 80% of browsers
    • Can send custom headers
    • Works with GET requests too
    • Supports older browsers
  • sendBeacon() is good when
    • Supported by 96% of browsers
    • You don't need much customization
    • You prefer a clean, elegant API
    • You don't want to compete with the browser's other priority requests

Avoid the "X-" prefix in Headers

Sometimes you see headers using an "X-" prefix, such as X-Auth. This "X-" prefix stands for "eXperimental" or "eXtension" and was long used to distinguish standard headers from non-standard headers.

Because the "X-" prefix came to cause more problems than it solved, RFC6648 says to avoid using the "X-" prefix. One problem caused by using the "X-" prefix is that when a non-standard header becomes a standard, the cost of handling the change becomes large.

What is 308 Permanent Redirect?

What_is_308_Permanent_Redirect.png

  • When a URL is redirected, 301 or 302 redirects while unconditionally changing the method to GET instead of keeping the original request method (GET or POST),
  • and 307 or 308 redirects while keeping the original method (GET, POST).
  • Of these, "308 Permanent Redirect" means the redirect is permanent, not temporary.

Troubleshooting

Invalid Content-Length header

This problem occurs because certain web browsers send an extra 2 bytes (a carriage return and a line feed) at the end of the message body (the data part) of an HTTP POST request. However, the web browser does not count these extra 2 bytes in the HTTP Content-Length header, so the message body contains 2 bytes more than stated in the HTTP Content-Length header.

The Web Proxy service in ISA Server 2000 includes a check for this condition, so when a web browser sends an invalid Content-Length header, a "400 Bad Request" error message is returned.

App does not update to the latest version after deployment unless the browser cache is cleared

See Cache Busting.

Documentation

RFC 2068
rfc2068 : Hypertext Transfer Protocol 1.1 standard (draft)
RFC_2068_KOR.txt
RFC 2616
RFC_2616_KOR.txt
RFC 2109
HTTP State Management Mechanism
Learning about cookies and sessions
HTTP/3 explained
https://daniel.haxx.se/http3-explained/
https://http3-explained.haxx.se/ko/
Http3-explained-ko.pdf

See also

Favorite site

Project

Article

Videos

References

  1. HTTP_multipart_form-data_raw_data.pdf ↩

  2. Popit.kr_-_I_just_did_not_know_HTTP2.pdf ↩

author Zer0 sitemap