Tag: RESTful Architecture

Journey to RESTfulness – Part 3 of 4 – Elements of RESTful Architecture

July 7, 2014 by Abhi·Comments Off

In this post we will talk about elements of RESTful architecture. However before we do that I would recommend reading the previous part of the series

Components

In REST, the various software components that interact together are known as software components. These could be considered as units working together in a RESTful design. These components are organized by the role they perform in the system. Dr. Fielding has defined 4 major component types in his dissertation.

Origin Server – is the component that listens for requests and provides responses. These responses could also be representation of resources as well. The origin server own the url namespaces for all its resources.

Examples of origin server include IIS, Apache, etc.

User Agent – is responsible for initiating a request or a state transition for a resource.

The most common example of a user agent are web browsers.

Gateway – sits between the origin server and user agent and provides additional processing like caching, load balancing, etc. Gateway represents multiple origin servers to the network. There can be multiple gateways on the network between the origin and user agent.

Examples of gateway are Synapse, Squid, etc.

Proxy – sits between the origin server and user agent and provides additional processing like caching, load balancing, etc. A proxy represents multiple user agents on the network and the client could determine whether to use or not to use proxy.

Examples of proxy are CERN Proxy, Netscape Proxy, etc.

Connecters

Connectors could be thought of as interfaces implemented by components to work. Connectors are organized by the role they play for the component. A component can implement multiple connectorsThe various types of connectors are:

Server – is the connector that listens for requests and provides responses. These responses could also be representation of resources as well. The origin server own the url namespaces for all its resources.

Examples of server include Web Server API, etc.

Client – is the connector that starts the resource request or resource state transformation requests.

Examples of client connector include Http library, etc.

Cache – connector manages the storage of resources and states of resources which could be used for specified time. This could be located at client or server and reduces lag time for requests.

Examples of cache include Browser cache, etc.

Resolver – transforms the resource identifiers into whatever is the address format (ip address, hostname, etc) so that the 2 components could make a connection. The advantage of using a resolver is that is provides deviousness between components to a level. This increases the lifetime of component references with changes in network topology, etc.

Examples of resolver include bind DNS lookup, etc.

Tunnel – connector relays request across a boundary. Any component could switch from active behavior to tunnel behavior.

Examples of tunnel SOCKS, SSL after HTTP CONECT, etc.

Resources

Resources are concepts and ideally should never change. These resource maps to entities and these entities or mapping could change overtime.

Examples of resources are video on YouTube, image on flicker, wall on Facebook, etc.

We could understand it with a simple example. We have resources or concepts like Employee, Manager, Developer and Trainee as show in the image below. We should remember that these concepts could still exist even when they do not have any actual entities mapped to them.

Map Concept To Entity

Let us introduce 2 entities: John is a manager in the company and Steve is a trainee. This will introduce a mapping between the concepts and entities John and Steve. Also John and Steve have a conceptual identities of their own as well. It would look like below. John is an employee as well as a manager.

Map Concept To Entity

Now say after few months the company decides to hire the intern (Steve) and designate him as developer. The concepts will remain the same but this will change the mapping of the concepts with entities. The employee resource now points to 2 different entities. Also let’s assume that the company gets another intern named Nick. While the concept of intern has not changed but still the mapping of resource to entity is now with Nick instead of Steve.

Map Concept To Entity

So the key point to note here is that resource is a concept in the system and it should be a stable concept. However what could change overtime is the mapping of these concepts with entities that forms the value for the resources. It is a common mistake to design the resources as entities but a costly one.

Resource Identifier

Since resource is a unique concept in a system then it is only logical to have a unique resource identifier for a resource. The server uses the resource identifier to make the resource available. As we discussed in the previous section that the resource should be stable concept which also implies that the resource identifier should not change frequently however the mappings that provide the value might change.

Now when I am designing a RESTful system for working over http then we will need resource identifiers for all the resources we saw in the previous section and they will look something like below. We have urls identifying resources. We have created hierarchy with employee at the top followed by the manager, developer, intern as well as each employee.

Resource Identifier

Resource Metadata

Along with the application data the resources also include the information specific that describes the resource itself. Metadata provides additional information such as url, links, location information, alternate resource identifiers for different formats or entity tag information about the resource itself. In case of HTTP this information is available in the headers. For example in the image below we could see the ETag information which is the metadata.

Resource Metadata

Representation

A representation is a concrete counterpart of the resource (concept) as it represents that concept at a point in time. There can be any number of representations for any given resource. In the modern web environment the same resource might be in different representations for serving different types of user agents. A representation is nothing more than a sequence of bytes.

Content negotiation is a process of selecting the best representation of a resource. The two categories of content negotiation are

Server driven content negotiation – In this type of content negotiation the server decides the representation of the resource depending upon the information available in client request.

Agent driven content negotiation – In this type of content negotiation the server and the agent work together to determine the best representation. Generally the server provides the user agent with multiple choices as links to those representations and then the agent chooses one of those representations.

In the Http request we could specify the type of content we are expecting.

Content/Type: application/json

Representation Metadata

Just like the resource metadata it also describes the representation and in case of HTTP this information is available as part of the HTTP Headers. It helps the clients and server to determine the course of action of the byte sequence (representation).

We could see in the image below that in the Request Headers are accepting contents of type text/html, application/xml, etc. and hence we have received the content of one of those types (application/xml).

Representation Metadata

Control Data

This defines the purpose of a message between components, such as the action being requested. Control data exists for both request and responses in Http.

Example of control data are If-Modified-Since, If-Match, If-None-Match

Control Data

Hypermedia

Hypermedia is all about reducing the coupling between a client and a server. This decoupling specifically refers to the client not knowing all the urls that are exposed a service. So ideally a client should be aware of only the entry point url of the service and then should be able to dereference all the other links of the service based on requirement.

Journey to RESTfulness – Part 2 of 4

July 1, 2014 by Abhi·Comments Off

In this post we will derive REST from constraints. However before we do that I would recommend reading the 1st part of the series here.

Deriving REST

Dr. Fielding in his dissertation talks about the method that he would use to define REST. This method is more constraints driven rather than requirements driven. A constraints driven approach identifies the factors that influence system behavior and then we apply the design so that the constraints works with those factors rather than working against them.

Requiements Vs Constratints

Many software architectures are built and designed for small set of requirements and as we get new requirement we grow our design to incorporate those. The PC architectures follow this pattern because the domain of the architecture and domain of the business are closely coupled. These designs solve programmer problems like encapsulation. These designs are designed and tested in a limited environment and then deployed at production where we discover that there are limitations that keep these designs from being broadly usable apart from the environment for which it was designed.

REST was designed to solve this problem by determining these constraints in a distributed architecture that restrict the design to be usable broadly. Then REST applies these constraints on a working design and thus shaping it incrementally. Hence we end up mapping the business domain on the architecture domain.

So as we conclude that REST is defined as the identifying the forces that are barriers in distributed computing then knowing these barriers might be helpful in understanding the significance of the individual constraints.

Fallacies of Distributed Computing

These are the set of assumptions that L. Peter Deutsch at Sun Microsystems (now Oracle Corporation) originally declared and it states the assumptions that the programmers unaccustomed to distributed applications invariably make. These assumptions ultimately prove false, ensuing either the failure of the system, a considerable reduction in system scope, or in giant, unplanned expenses needed to revamp the system to satisfy its original goals.

The 8 Fallacies of Distributed Computing are as below:

The network is reliable.
Latency is zero.
Bandwidth is infinite.
The network is secure.
Topology doesn’t change.
There is one administrator.
Transport cost is zero.
The network is homogeneous.

So we should design our architecture to work with these forces of nature rather than against them.

Constraints

Let’s have a look at few architectural constraints that define the RESTful style.

Client – Server constraint

This is one of the fundamental constraint and enforces the constraint in for the client server architecture. The constraint defines all the communication between nodes in a distributed architecture as being between a client and a server. A server is continuously listening for message and when a client sends a message to the server then the server processes it and returns a response. This constraints allows separation the concerns of server and client mainly for User Interface and thus allows different types of client to work with the server and also the client can evolve independently of the server.

Client Server

The guiding forces for the Client-Server constraint are as follows.

Network security is improved as by scoping the connections between clients and servers we can make the system more secure.
Administration is easier as by scoping the connections between the clients and servers we limit the responsibilities of client server and hence they are easy to manage.
Heterogeneous network is workable by connecting and disconnecting any number of clients on multiple platforms with no impact on the server

The properties of this constraint are:

Client portability is more because the client structure is independent of the server
Scalability is better because the server does not have to worry about the user interface details
Independent client evolution happens as the server and client are independent.

Stateless constraint

In distributed application the stateless constraint is quite prominent. Stateless constraint does not imply that we should maintain no state of the application but Stateless constraint applies to the communication between the client and server. So the client server interaction must be stateless so that the server is able to process the request with just the information provided by the client request without any context available on the server. The design with Stateless constraint will imply that the state is stored on the client. This design is quite suitable in designs where clients and servers are constantly being added, removed or their network identities are being modified.

Statelessness

The guiding forces for the Stateless constraint are as follows:

Network Reliability is improved by storing the state in the client and we allow the interaction between the client and server to be stateless and this give the application the capability to recover from network errors.
Network Topology will be simpler since the state of the client is on the server we can add, remove clients and servers from the network without any corruption of data.
Administration will be simple when we have stateless interactions.

The various properties of this constraint comes are:

Visibility is improved since the system does look for any further than the current request so the full nature of the request is known easily.
Reliability of the system is more reliable because the system could recover from partial failures.
Scalability is better because the server does not have to worry about the state maintenance across various requests and servers.

There are a couple of design trade-offs that we would have to do when following this architecture.

Network Performance might decrease as we might me sending more or repetitive data in each request for the server to have enough information to process the request independently.
Client consistency might be lost as the state management is done on the client and the implementation might be different on different platforms.

Cache constraint

According to REST the response from the server should be implicitly or explicitly labeled as cacheable or not cacheable. When the response is cacheable then the client is allowed to reuse the response in equivalent requests. This could allow our applications to reap the benefits of caching at multiple levels (server, intermediate or client). This will majorly improve the network efficiency.

Cache

The guiding forces for the Cache constraint are as follows:

Latency is reduced as some the requests might be served on the client itself and some of them from other caches.
Bandwidth consumption is less since some requests might not even reach the server and served beforehand by cache.
Transport cost is reduced as the number of requests might be reduced.

The properties of Cache constraint are:

Efficiency is improved since the application might have less latency and sucks less network.
Scalability is improved since the application is more efficient it could handle more clients.
User perceived performance could be improved when the response from the request is coming from the cache.

The design trade-offs that we might have live with, in this architecture is

Decreased reliability on data if the data is stale and differs significantly from the one which would have been provided from the server (if requested).

Uniform Interface constraint

This is the major differentiator between the REST architecture and other network-based architectures. This constraint emphasizes on having a Uniform Interface for all the components in the architecture and could be achieved by applying the generality principle to the component interface and hence simplifying the overall system architecture and improving the visible interactions. So each component talks to the other via standard mechanism. Implementation of decoupling from the service could lead to independent evolution.

Uniform Interface

To achieve the Uniform Interface constraint we need to include the following elements in our design:

Identification of resources
Manipulation of resources through representation
Self-descriptive messages
Hypermedia as an engine of application state (HATEOAS)

The guiding forces for the Cache constraint are as follows:

Network reliability is improved when all the components of the design understand the message sin the same way.
Network topology could be simpler and evolve as the clients and serve communicate with each other following the same interface
Administration could be easier since we could introduce generic tools for network optimization
Heterogeneous network could be supported better because the communication interface is the same between different components.

The properties of Uniform Interface constraint are as below:

Visibility is more when we are exchanges the same Interface between all the components of the architecture.
Evolvability for each component will be easier as all the component talk the same language

The design trade-offs that we might have live with, in this architecture is

Decreased efficiency since the data will be transferred in standard format rather than the specific format in which it is needed by the application.

Layered System Constraint

Layered system constraint states that a component in a system should only know about the components of the layer with which it is interacting.

Layered System

The guiding forces for the Layered System constraint are as follows:

Network topology could be simpler as the communication is restricted to the layers and when we change the network components then the only the elements that interact with that layer will be impacted.
Security will be better since we layering will allow us to place trust boundaries in layers know the possible components interaction.

The properties of Layered System constraint are as below:

Scalability is enormous when we have layered system and modern web is a living example of this.
Manageability is also great since each layer could be managed by different admins and still be perfectly operational and scalable. Example my browser know to manage the connection proxy which is managed by my company which know how to connect to Internet which is managed the ISP and so on and so forth. Each layer is managed by different system with different policies.

The design trade-offs that we might have live with, in this architecture is

Increased latency since the data might travel more layers as each component will be communicating with the layer it’s supposed to as compared to a direct connection. We can mitigate this trade off by usage of shared caches and intermediate load balancers.

Code on Demand Constraint

This is listed as an optional constraint in Dr. Fielding paper and this might be one of the reasons why it’s not talked about as much. Code on Demand states that along with provides the clients with the data and metadata, the servers could also provide executable code. The idea is to provide the client with readymade features so that they do not need to write or rewrite them.

Code On Demend

The properties of Layered System constraint are as below:

Simplicity is increased since the client have less number of pre written features and these features could be made available by the server.

The design trade-offs that we might have live with, in this architecture is

Reduced visibility since the clients are downloading the readymade code and features and these might affect caching, manageability and security. So the key rule to applying this constraint is that we should apply this constraint is such a way that the clients who support it should be benefitted by it and the client who do not support this should not break.

Any questions, comments and feedback are most welcome.

Journey to RESTfulness – Part 1 of 4

June 30, 2014 by Abhi·Comments Off

I have been learning and working on REST for a while now. But I have on many blogs that there are disconnects between what REST actually is and what is perceived. So I wanted to write an article based on my understanding of REST. I would talk here about what REST actually is and how to design systems that follow the principles of REST. I will talk about following things in the series.

Components of modern distributed architecture
Properties of RESTful design
What REST is and what it is not?
The journey to RESTfulness
REST and the rest
RESTful Architecture
Elements of RESTful Architecture
Designing for RESTful Services
REST and Cloud

Components of modern distributed architecture

Distributed application development is more challenging in the modern times as we are dealing with of everything users, services, hardware, etc. Few of the major problems that we face today are:

Interoperability between heterogeneous applications

In simple words we want to integrate different applications which have been developed with different frameworks and might even run on different platforms. You must have seen multiple ways to sign up or share various website. It’s a live example of integration of heterogeneous applications in one place. These are different service providers who could not make assumptions for the applications and services provided by any other platform and yet we would like to use all these providers at one place.

Signup

We want these different integration pieces to simple, consistent and reliable.

Heterogeneous

Diversity in Devices

REST is based on the idea of a network based API rather than a library based API and this goes hand in hand with integration of heterogeneous applications and services available. Today we want the integration to be device independent. When we say devices we not only mean smartphones and tablets, it includes most of the electronic devices including cameras, navigation devices, watches, car in dash and what not.

Device Integration

We want our services and applications to work seamlessly for all the devices. Since most of these devices run native apps and not web based applications maintenance and updates are big challenges. And you could imagine the issues we might come across for performance and efficiency when working with multiple devices because of the network availability and amount of data that can be transferred on the network (data being paid as per usage).

To be or not to be: Cloud

Apart from the interoperability between different services and different devices a major problem that we face is the number of users simultaneously accessing the service. We could most out of the scalable infrastructure if have a scalable architecture. The various organizations have already been taking the advantage of the elastic infrastructure provided by various companies. The elastic infrastructure allows the business to automatically grow or shrink the computing power and storage capacity of the applications according to the number of users and pay only for the resources that are used.

Cloud

However we need to understand that cloud just provides the hardware and capability to scale our applications and services and we need to develop our applications and services in a way so that could utilize the various capabilities offered by cloud. We need to build saleable architecture to take advantage of saleable infrastructure.

We could design scalable architectures by neither depending on the middleware in our infrastructure nor on the hardware does that have inherent limitations of its own. We should build applications with transparency in mind so that in case of any errors or failures we don’t have to dig through for days.

The gives us the capability to eliminate the situations shown on the left and provide us a clean (that’s all you need to think) and maintenance free hardware.

Server Rooms

Properties of RESTful design

There are various properties of REST design align with the solutions of the challenges that we discussed above.

Heterogeny – The ability to seamlessly interoperate with other participants regardless of language or platform.
Scalability – The ability to limit complexity between components in a distributed system, efficiently handling requests and scaling out horizontally when needed.
Evolvability – The ability for client and services to evolve independently of one another.
Visibility – The ability for value added components such as intelligent gateways to operate correctly without needing access to any hidden or proprietary state such as session state.
Reliability – The ability for clients to recover more reliably from failures by developing rich compensation strategies.
Efficiency – The ability for multiple components such as proxy servers and caches to participate in the handling of requests taking load away from your server.
Performance – The ability to use caches, greatly improving the speed in which a response can be delivered, giving the impression of increased performance.
Managability – The ability for simpler management due to interactions between components happening in a highly consistent and visible way.

One or more of these properties align with solving each of the challenges that we discussed previously.

What REST is and what it is not?

REST is not RPC – In RPC the design target of a network interaction is a remote function and the goal RPC of RPC to abstract all the network details so that the developer writing the code should not care about the components interacting over the network.

Whereas in REST the design target of a network interaction is a network resource. Also the network schematics are part of the design.

REST is not just HTTP – Http is the underlying architecture on which REST is based but using just HTTP verbs correctly does not make our services completely RESTful. However most RESTful systems use HTTP as the underlying platform

REST in not just URI – URIs hold an important place in RESTful design but extreme focus on URIs could push us back to thinking designs more the RPC way.

REST is not just anything that is not SOAP – SOAP is more of an implementation detail and REST is more like an architectural style. SOAP aligns iteslf with RPC design style and anything which not SOAP does not imply that it is REST.

Representational State Transfer better known as REST is an architectural style defined in the dissertation of Dr. Roy Fielding at University of California, Irvine in 2000. He designed REST for larger architectural concepts on which web was designed. As per fielding the phrase “Representational state transfer” represents for how a well-designed application behaves as virtual state machine of web pages where progress is made via links.

The journey to RESTfulness – Richardson’s Maturity Model

This model gaining attention and importance in the community and has been referred by Martin Fowler and books like The RESTful CookBook. It is a model we could use to grade our API as per the constraints of REST. The more our APIs adhere to these constraints the closer they are towards RESTfulness.

The different steps in the image below represent the incremental steps towards REST. These are in no way the levels of REST.

RESTfull

L0 – represents that we are following the RPC style with the Plain Old XML (POX). This is the most elementary level of the service maturity.

L1 – represents the use differentiated resources.

L2 – represents the usage of HTTP verbs and HTTP status codes.

L3– represents the use of hypermedia controls.

More details on the Richardson’s Maturity Model could be found at Martin Fowler’s blog.

Any Questions, Comments and feedback are always welcome.