SOA based Search Design - Handling of bulk data on server side
Its me again, For past few days my life is all about searching, search criteria and analyzing its result. The product I work has to scale up-to returning search results of the magnitude of few hundred records. Unfortunately we don’t have the luxury of having the pagination for these search results, hence there is a high amount of tuning required on the server side.
In this article I will be describing a SOA based search design for handling bulk data in a multi layered architecture; The project I was working was architect to have 3 layers: Facade Layer, Services Layer and a DAO Layer.
Facade layer was implemented though EJB 3 and as expected was the main communication interface for the client, our clients talk to the server side Facade though the Data Transfer Objects (Dto); One of the main responsibility of the facade is to convert Dto objects passed by the clients to the Business Object (Bo) for request processing and during response scenario, facade also converts the Business objects to the Dto and send the response to the client.
Service Layer is the place where the main business logic will placed,the communication to/ from the services layers always happens through the Business Objects. Main responsibilities of this layer include invoking the Data Access Object Layer (Dao) and invoking other Services.
Data Access Object Layer is mainly responsible for invoking the persistence api (most of the cases, it will be ORM like hibernate, JPA).
To achieve the flexibility in the design approach, Dto are used to communicate between the client and server, but it has to be noted that all the processing on the server side happens on Business Objects.
Considering this architecture, Even for a search case too, search criteria will be passed to the server in the form of a Dto, and in the facade layer this search criteria dto will be converted to a Bo and then the criteria in the form of a Business object will be passed to the Dao and based on the persistence technology an appropriate query will be executed and the results in the form of the Bo will be returned to the facade, where again the Bo will be converted to the Dto objects.
Now, this approach works very well for low to medium volume search result sites; in order to optimize this search we have been looking around for some options and finally arrived at saying “For searches, why not allow the criteria (Dto equivalent) to be passed to the Dao layer and get the results from Dao in the form of Dto”. Essentially, all we are doing is eliminating the conversions from Bo to Dto and vice-versa. For large volume Search results, the conversation to/ from Dto/Bo makes a big differences (time consuming process). This approach also gives us a flexibility of selecting any specific attribute in an entity (unlike the case-of the Query though Business Objects, the entire Business Object will be loaded even though you need a single attribute of that entity)
There is already some support by JPA to support this kind of architecture for searches, for example look at the following query
select new com.alpha.beta.SearchResult(emp.id, emp.name, emp.salary) from Employee emp
Now the ‘SearchResult’ class should have a constructor matching to the above construct
i.e,
public class SearchResult{
private Long id;
private String name;
private BigDecimal salary;
public SearchResult(Long id, String name,BigDecimal sal){
this.id = id;
this.name = name;
this.salary = sal
}
}
If the query retuns a ‘List’, In this example the list will contain a list of ‘SearchResult’ Objects.
To Summarize the advantage of this approach (of bi-passing the layers for search)
- There is no extra conversation To / From Dto/ Bo.
- It also possible for some cases to load the attribute without loading the entire Business Object.
- This approach also leaves the developer to use native SQL for complex sql which may not be implemented through some Persistence API.
Written by Ravi Nallakukkala on April 1st, 2007 with
no comments.
Read more articles on Design.
- [+] Digg: Feature this article
- [+] Del.icio.us: Bookmark this article
- [+] Furl: Bookmark this article