Consolidated Design Rules and Standards for Great REST API - Part 1

In a previous thread named Consolidated Design Rules and Standards for Great API, we explored the merged list of rules and standards for designing Web APIs which can be applied for REST and non-REST APIs. This thread builds on the previous thread and deeps dive into REST APIs in particular and lists the most common conventions, rules and standards that make REST APIs great and will be grouped by the HTTP methods.

At the end of this thread, we will see a step-by-step process to help you design a Great REST API.

What REST really means?

To understand what REST really means, we should first understand what Resource, Representation, Application State, and Resource State mean.

Resource

A resource can be anything that can be stored on a computer; a file, a record in a database, a computational algorithm, etc.

Resources need an identification mechanism to give them a unique id that distinguishes them from any other resource. In our case, URLs are used to give the resources distinguished identifications.

Representation

As explained, a resource can be a record in a database which can be stored as a set of binary data. But transmitting database binary records between the consumer and server will be meaningless. It will be more realistic to transmit a document that describes the database record (or any other resource) in a machine-readable language. This document is called representation.

A representation is a machine-readable document used to describe a resource for the purpose of communication between the client and server (i.e., consumer and provider). Common representations formats are JSON and XML.

Application State

With web browsers, the user can open any page and he can surf the web, moving from one page to another. In this case, the currently open page is considered as the current state of the application (e.g., the application here is the browser itself).

In the world of APIs, if a consumer did a call to get a list of users and then did another call to get the details of one of these users. The application state, in this case, was the "list of users" and then moved to the state "detailed user".

The application state is maintained on the client, and the server doesn't know anything about a client's application state.

Resource State

If a server maintains a resource called user, whose name is Alex, and his locked flag is false, and somehow it has been updated to Sam and the locked flag to true. In this case, the user resource state has moved from "Alex, not blocked" to "Sam, blocked". that's what we mean by resource state, which is maintained by the service (on the server side). Clients can't maintain a resource state, but they can get a representation of the resource state.

REST

Representational state transfer (REST), as its name implies, it allows the server to transfer the application state by sending a representation of a resource (e.g., as a response to a GET request), and it also allows the client to transfer the resource state by sending a representation for a resource to the server (e.g., send a PUT request).

REST as an architectural style was introduced by Roy Fielding in 2000. He defined REST as a set of constraints on top of the word wide web to make the APIs look like the web. The fifth constrain (Uniform Interface) is the fundamental extension to build REST API:

Constraint 1: REST architecture must be built based on Client-Server architecture

Constraint 2: REST architecture must be a Layered system of proxies and Firewalls, and any other intermediates can be added and removed without affecting the communication between the client and the server

Constraint 3: REST architecture must support Cache

Constraint 4: REST architecture must be Stateless which means that the server should not know anything about the Application State (see the above definition)

Constraint 5: REST API must be built using Uniform Interface. To have a uniform interface, the following constraints need to be applied:

1. Resources must be identified in the requests using URIs
2. Resources must be manipulated by sending representations using a standard set of HTTP methods (GET, POST, PUT, DELETE, OPTIONS, HEAD, PATCH)
3. Messages (request and response payload) must be self-descriptive by using media types (for example, application/json and application/xml)
4. Hypermedia As The Engine Of Application State (HATEOAS) must be supported to allow the client to surf the API, similar to the end-user surfing the web.HATEOAS means the hypermedia returned by the server works as engines that move the client from one application state to another.

In this part, we will cover the generic rules; in the next part, we will cover the rules of the HTTP methods.

Generic Rules

The following rules must be followed regardless what is the HTTP method used:

In general, apply the Performance and Reliability rules defined in the previous thread.

Generic rules for URL naming

Rule 1: Apply the Security rules defined in the previous thread

Rule 2: Apply the Performance and Reliability rules defined in the previous thread

Rule 3: Apply the Clarity and Consistency rules defined in the previous thread

Rule 4: Apply the Change Management Rules defined in the previous thread

Rule 5: URI should identify one and only one resource

Rule 6: Resource can be either a collection or an object

1. Example for collection: /users
2. Example for single user object: /users/121

Rule 7: Use plural nouns for resource names. For example: /users

Rule 8: Resource URI MUST NOT end with (/). /users

Rule 9: Refer to a specific object for any resource as a sub-resource

1. Use the following to get a user with id 121: GET /users/121

Rule 10: If a resource is hierarchically related to another, it is better to be presented as a sub-resource.

1. You can use the following to get all the addresses for user 121: Get /users/121/addresses
2. Or you can use the following to get all addresses for all users: Get /users/addresses

Rule 11: Hyphen (-) can be used to enhance the URI readability:

1. Use the following to get the list of users types: GET /users-types

Rule 12: Avoid underscore (_) in URI

Rule 13: Don't use Uppercase in URI

Rule 14: Avoid Dot (.) in URI

Rule 15: Avoid comma (,) and semicolon (;) in URI

Rule 16: Avoid spaces in URI

Rule 17: Don't use verbs in URI for CRUD operations

1. DO NOT USE: GET /getusers

Rule 18: For non-CRUD operations (validate, calculate, check, search, lock, unlock, merge, copy, duplicate, process, etc.), use a verb as a sub-source:

1. It's ok to use the following to lock a specific user /users/121/lock
2. It's ok to use the following to lock all users /users/lock
3. It's ok to use the following to search for users: /users/search
4. It's ok to use the following to merge all the addresses of a specific user: /users/121/addresses/merge

Rule 19: For search operations that may need to fetch data from multiple resources, URI can be defined as a parent resource:

1. GET /search?q=abc

Rule 20: To define variables in URI, use standard URI Template format as per URI Template standard:

1. In the following, {userId} must be replaced by the consumer with a valid user Id value /users/{userId}/addresses

Rule 21: Query parameters can be used for filtration, sorting and pagination:

1. Filtration example; get all active users: /users?active=true
2. Pagination example; get page 5 where the page size is 10: /users?pageId=5&pageSize=10

Rule 22: Use (&) to separate query parameters

Rule 23: Apply one of the following versioning identification strategies:

1. URI Components (preferred option); before the resource (e.g., /v2/users) or after the resource (e.g., /users/v2). Endpoint Redirection pattern can be used to redirect consumers to the new version endpoint
2. Custom HTTP header. For example, API-version: 1
3. Request parameter, e.g., users?v=1

Rule 24: Avoid introducing new media types for each version because it may reduce API interoperability. The following examples are not recommended:

1. application/json;v1
2. application/vnd.usr+json;v1

Rule 25: Apply the following techniques to minimize the impact on the consumer in case of backward incompatible changes:

1. Introduce a new version for the new changes
2. Introduce Service API proxy design pattern for existing customers. API proxy retains the original API and contains logic to accommodate the changes.
3. In case of endpoint change, apply the Endpoint Redirection design pattern using the Location header to redirect the consumer to the new endpoint. In this case, the server must return the status code (301 Moved Permanently).
4. Apply the Termination Notification design pattern to announce version has been deprecated and announce the date of the old version's retirement. For this purpose, you can use Deprecation (see IETF-Deprecation) and Sunset HTTP headers. see RFC8594.
5. Build a change log to track the versions and changes applied in each version.
6. Once the old service is switched off, the server should return the status code (410 "Gone")

Generic rules for Data Models

Rule 26: You may support more than one data format, but make sure to keep JSON as your primary and default format

Rule 27: If multiple formats are supported, allow consumers to negotiate for a given format using Accept header

Rule 28: Give meaningful entities and fields names and try to use standard vocabulary by consulting the following standards (schema.org, GeoJSON, Microformats wiki, Dublin Core, Activity Vocabulary, FOAF)

Rule 29: Define a schema for the data model (JSON Schema or XML Schema) and Assign the right data type, and don't assign string data type for everything unless it is required. In the case of JSON, describe your API using Open API Specs and use $ref to include data models schemas.

Rule 30: Fields names must be camel-cased, starting with a lowercase "givenName"

Rule 31: Fields names must start and end with a char

Rule 32: Fields names must use only ASCII alphanumeric characters

Rule 33: Ensure to reduce the quantity and restrictiveness of validation logic embedded in the API and keep them generic. Detailed validations in the API can make it coupled to the underlying implementation, and accordingly, it will always be eligible for change whenever the underlying implementation is changed. See the Validation Abstraction design pattern for more details.

Rule 34: Use standards for field values. Use ISO3166 for countries, ISO4217 for currencies, RFC3339 for dates and times formats, etc.

Rule 35: If you have created a custom list of media types (although not recommended), your schema must be assigned one of the predefined media types. For example

1. Use JSON schema can be assigned vnd.user+json
2. Use XML schema can be assigned vnd.user+xml

Rule 36: Avoid encoding binary data within the data models using base64. instead, use the following:

1. Multipart media types
2. Separate links to fetch the binary data

Rule 37: Make sure to support hypermedia in the representations for the following purposes

1. To link entities/resources together For Entity Linking purposes (e.g., related, self, parent, author, subsection, etc.)
2. To support the navigation for collection pages (e.g., previous, next, first, last)
3. To support the move between workflow activities (e.g., start, edit, next, previous, etc.)
4. To support the retrieval of binary data (in case multipart media types are not supported)

Rule 38: Build standard schema(s) using supported data formats (JSON, XML, etc.) for Hypermedias details. Below is a recommended format:

{

"href": "URI or URI Template for linked entity",

"rel": "URI used to identify the relationship type between this entity and linked entity e.g., parent, child, next, previous, etc."

}

Rule 39: For a list of standard relationship types, refer to Web Linking standard and Link Relations, RFC5005, RFC4685, RFC5829, Microformats Wiki for a list of registered relations types and make sure to use lowercase characters for all relation types

Rule 40: In each representation, include a self-link as follows.

{

"id": 121,

"userName": "H1212",

"firstName":"Haitham",

"links": [

{

"href": "/users/121",

"rel": "self"

}

]

}

Rule 41: Include the link in a consistent manner within your data model. Below "links" is a recommended approach to support entity linking:

{

"userName": "H1212",

"firstName":"Haitham",

"links": [

{

"href": "/users/121",

"rel": "self"

},

{

"href": "/users/12",

"rel": "parent"

},

{

"href": "/users/121/addresses/3",

"rel": "related"

}

]

}

Rule 42: Avoid removing the links or changing the link relation type because that is not a backwards-compatible change, and API consumers will be affected.

Rule 43: Collections models must provide self-link for the collection resource itself

Rule 44: Collections entries must also include self-links pointing to the individual resources. This will allow the client to get more details about the individual resources.

Rule 45: Define a standard structure for Collections. For example:

{

"users": [],

"links": []

}

Rule 46: Collections should support pagination by providing the total number of pages, linking to the next page and link to the previous page and preferably two links for the first and last pages. The following is a response for /users?pageId=5&pageSize=10

{

"users": [

{

"userName": "H1212",

"firstName":"Haitham",

"links": [

{

"href": "/users/121",

"rel": "self"

},

{

"href": "/users/12",

"rel": "parent"

},

{

"href": "/users/121/addresses/3",

"rel": "related"

}

]

}

],

"links": [

{

"href": "/users?pageId=5&pageSize=10",

"rel": "self"

},

{

"href": "/users?pageId=6&pageSize=10",

"rel": "next"

},

{

"href": "/users?pageId=4&pageSize=10",

"rel": "previous"

}

]

}

Rule 47: Build standard schema(s) using supported data formats (JSON, XML, etc.) for error reporting and use them across all the APIs for failure conditions only.

1. One option is to use the standard error details schema proposed in RFC7807
2. Another option is to build your own custom structure and standardize it across all the APIs (e.g., note that the below structure supports reporting multiple errors):

{ "errors": [{

"code": "long-form error code e.g., name_too_long",

"busMsg": "Describe the error from a business point of view, and the value should follow the same language requested using the Accept-Language header. e.g., Name provided is longer than 16 char",

"tecMsg": "Describe the error from a technical point of view. E.g., Name length must not be greater than 16",

"note": "Note for the consumer developer. E.g., Please refer to the following URL for more details: http://apidoc.com"

"recommendation": "Optional field for a recommendation. E.g., minimize the value length and submit again."

]}

}

Rule 48: return longer-form response codes in the error report payload (see previous rule)

Generic rules for Media-Types

Rule 49: Avoid Application-specific media type (e.g., application/vnd.user+json). Defining custom media types reduces API interoperability. But if you had to define custom media types, make sure to document them and map them with the predefined data models as part of the Contract-First design approach (see the previous thread for more details). Most importantly, avoid introducing new media types for each version.

Rule 50: You may support more than one data format, but make sure to keep JSON as your primary and default format

Rule 51: Allow consumers to negotiate for a given format using Accept header

Rule 52: Avoid encoding binary data with the data models using base64. instead, use multipart media types

Rule 53: UTF-8 must be the only charset for request and response payload. UTF-8 is already the default encoding for both media types, application/xml and application/json, which should never be changed.

Generic rules for Metadata

Rule 54: Meta-data should be transferred using HTTP headers only

Rule 55: HTTP headers should be used for meta-data only and shouldn't be used for business-related details

Rule 56: Return Content-Length header must be used in the response

Rule 57: If the request has a payload but doesn't include the Content-Type header. Return status code (400 Bad Request).

Rule 58: If the request has an unsupported media type defined in the Content-Type header, return status code (415 Unsupported Media Type)

Rule 59: If the request has a body, but it doesn't match the media type defined in the Content-Type header, return status code (400 Bad Request)

Rule 60: If the request has a body and it matches the media type defined in the Content-Type header, but the message structure or values are invalid, return status code (400 Bad Request)

Rule 61: In response, return the payload using the same format used in the request payload (e.g., request Content-Type). If the request has no payload, use the Accept header to allow the consumer to negotiate for his preferred data format (e.g., JSON, XML, etc.). If no Accept header is provided in the request, use the default data format (e.g., JSON).

Rule 62: Content-Type header must be used in the response and must match the response body payload (for example: don't return application/json while the response payload is XML-based)

Rule 63: Accept-Language header must be supported to allow the consumers to request the business details in a specific language if multilingual resources are maintained

Rule 64: if the Accept-Language header is not provided, return the response using the default language (e.g., en)

Rule 65: Encourage your consumer to use the Content-Encoding header in the request to compress the request payload

Rule 66: Support the Accept-Encoding header to allow the consumers to get a compressed response payload compression.

Rule 67: Return the Content-Encoding header in response to inform the consumer whether the response payload is compressed or not

Rule 68: Encourage your consumers to send the Date header, and the API must always return the Date header in the response in all cases, success or failure.

Rule 69: Support the Authorization header for the OAuth access token and make sure to validate it with the OAuth provider

1. Make sure the access token is not expired.
2. Make sure the access token is permitted to perform the requested HTTP method on the requested resource

Rule 70: If the consumer didn't provide or provided invalid credentials, it is recommended to return the WWW-Authenticate header to tell the client what kind of authentication is expected

1. This header must accompany with HTTP Status code (401 Unauthorized)

Rule 71: Return Retry-After header in the response if the consumer exceeded the maximum limit allowed.

1. This header is usually accompanied by the HTTP status code (429 Too Many Requests)
2. It is also recommended to return the following custom headers as well X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset

Rule 72: If the consumer requested an unsupported method, return the HTTP Status code (405 Method Not Allowed) and return Allow Header to inform the consumer of the supported methods

Rule 73: If the User-Agent header is provided, use it for logging and statistical purposes only

Rule 74: Use the Link header to provide a URL for human-readable documentation describing the response payload structure. In case of failure, return, in the Link header, a link to a human-readable document that informs the client how to correct or debug the error. The profile is a standard link relation as per RFC6906

Link: <https://dev.twitter.com/docs>;rel="profile"

Rule 75: If you want to define a new custom header, do not use X- for "extension" as a prefix.

1. Using X- was an old recommendation, which is not valid anymore. check RFC6648

Rule 76: Return the X-OAuth-Scopes header (inherited from GitHub APIs) to list the scopes for which a token has been authorized

Rule 77: Return the X-Accepted-OAuth-Scopes header (inherited from GitHub APIs) to list the scopes that the action requires

Rule 78: Return RateLimit-Limit to inform the consumer about the maximum rate allowed

Rule 79: Return RateLimit-Remaining to inform the consumer about the remaining available requests before reaching the max limit

Rule 80: Return RateLimit-Reset to inform the consumer when the current limit window will be reset

Rule 81: Use the Signature header to sign the request and response payload. The value should consist of the signature timestamp (to prevent replay attacks) and the signature value. Here is an example:

Signature: t= 1613907779, v=erwewrwrwerwerwerwrwerwwfsdfsd123wqdad31=

Rule 82: Don't use HTTP headers to customize the HTTP method's behaviour

Generic rules for HTTP methods usage

Rule 83: Avoid using nonstandard custom HTTP methods

Rule 84: Don't use the HEAD HTTP method. It is useless

Rule 85: Don't use TRACE and CONNECT; they are only applicable to proxies

Rule 86: LINK and UNLINK methods are not yet standardized, and you shouldn't use them

Rule 87: Don't use WebDAV methods, as they are not widely supported.

Rule 88: Use GET for reading purposes or for performing safe non-CRUD actions (e.g., validate, calculate, search, compare, etc.)

1. Safe actions mean no updates on the resources states will happen when calling the service.

Rule 89: Use POST to create new parent objects or to execute unsafe non-CRUD operations (such as lock, unlock, merge, copy, duplicate, process, etc.). Such activities are sometimes called tasks, and the resources handling tasks are called controllers or processors.

Rule 90: Use PUT to update an existing object or insert a sub-object for an existing parent object

Rule 91: Use DELETE to delete an existing object

Rule 92: Use PATCH to Modify part of a resource.

Rule 93: Use OPTIONS to retrieve the resource's allowed methods. It is good to support it, although it is not popularly used.

Rule 94: Use each HTTP method as per its semantics specified in HTTP specs. Don't try to override HTTP method behaviour.

Generic rules for HTTP Status Codes

Rule 95: Don't use HTTP status codes (301 Found)

Rule 96: Return HTTP Status code (429 Too Many Requests) if the client exceeds the maximum rate limit allowed

1. In this case, it is recommended to return the Retry-After header with a value that informs the consumer when he should try again.
2. It is also recommended to return RateLimit-Limit to inform the consumer about the maximum rate allowed.
3. And RateLimit-Remaining to inform the consumer about the remaining available requests before reaching the max limit.
4. And return RateLimit-Reset to inform the consumer when the current limit window will be reset.

Rule 97: Return HTTP Status code (500 Internal Server Error) if the client did everything correctly but the server could not handle the request because of an internal error. Don't use 500 if it is a client's fault.

Rule 98: For 4xx errors, it is recommended to return an error payload in the response body to describe the client's mistake.

Rule 99: For 5xx errors, it is recommended to return an error payload in the response body to tell the client about the issue and if there are any recommendations (e.g., ask the client to retry again later)

Continue to Part 2