Review: Planning the Review, Part 2
Use of Technology
There are many vendors and software applications which facilitate the management, review and production of electronic evidence. The use of these constantly evolving technologies and services should be considered for the following key phases of the review process:
De-duplication / Scope Reduction
Reducing the volume of data in a review collection translates directly into cost savings. One of the best methods of data reduction is a process known as de-duplication. For instance, an email message sent to three people in an organization could potentially exist in more than 10 locations within the company's systems (each user's computer, the Email server, backup tapes of email server, etc.). De-duplication technology allows duplicate email messages and other files to be tracked and reduced to one item in the review set. Because the file only appears once, it is only reviewed once, which dramatically reduces the billable hours spent on the review. De-duplication also facilitates a more consistent and accurate production, as the item can only be marked one time as "privileged" or "responsive" without the risk of another reviewer marking a copy differently.
As previously discussed, there are many ways to conduct a review of electronic data. Regardless of the format (native, HTML, paper or TIFF/PDF), the key is to get the review collection into a uniform state with the ability to move from item to item quickly and efficiently. Because the data gets processed, it can quickly be presented to the reviewer without the need to open the item in its native application. (See the Processing chapter.) For instance, one could instantly go from an email message (.MSG) to a Word document (.DOC) to an Excel spreadsheet (.XLS) with the click of a button. Converting, or "processing," the data also makes it possible to "tag" or annotate documents with the issues and subjective codes set forth in the review objectives. (See the Review Objectives section.) Additionally, the use of data processing technologies facilitates redaction or confidential information and production / bates numbering of responsive documents. (See the Production chapter.)
Once processed, electronic data can be easily searched for names and terms. Search techniques can be applied to exclude, prioritize and organize data, thus making the review more efficient. For instance, a search of attorney names would create a review set likely to be privileged. Rapidly evolving technologies such as concept searching, visualization and data grouping tools can help reviewers get through documents faster. The following examples illustrate the scope and benefits of these new tools:
- Concept/Context Searching - can bring issues to the surface and identify unknown items related to the case or bring similar documents together.
- Visualization Tools - show documents in a way to let the reviewer manage the information or groups of information in a more efficient way.
- Near Duplicates - indicates documents which are almost alike such as different versions of an excel worksheet.
- Email Thread Management Tools - lets the reviewer see all the threads of the same email chain and apply consistent coding across all iterations.
- Linguistic Experts - special companies who use advanced language analysis to find and group important documents.
New technologies continue to evolve and are becoming generally accepted throughout the industry. While they can potentially lead to cost savings, each situation needs to be evaluated to determine if it is the right solution. (See the Selecting a Vendor and Emerging Technologies sections in this chapter).
Selecting a Platform for Review
Choosing an appropriate platform for the review is an essential step in planning your review. There are many options available: in-house solutions, ASP or web-based solutions, native file reviews, TIFF/PDF file reviews, etc. The important thing is to understand your objectives and select a platform that will optimize your ability to reach those objectives. The Selecting a Vendor section provides an in-depth look at these options, as well as questions you should pose to the vendors to properly evaluate their capabilities.
Understanding Production and Delivery Requirements
The proposed rule changes to Federal Rule of Civil Procedure 26(f) require that opposing parties meet and confer about the format of production early on in the planning stages. Agreements made during the meet-and-confer process will influence the review objectives, techniques used, platform chosen and the overall project workflow and milestones.
Both parties must have plans to manage (i.e., how to bates number/redact/confidential stamp native files) the agreed-upon production format within the time constraints allowed by the court. See the Production chapter for more in-depth information around the production process.
Factors to consider regarding the production timetable during the planning stages:
- Overall volume of data to be reviewed and complexity of the issues being reviewed.
- Converting to tiff may require additional time once review is complete - are rolling productions acceptable?
- Volume of redactions and the workflow provided to perform them.
- Ability of vendor to produce to requirements. A sample run should be performed well in advance of any deadlines.
Time Considerations/Resource Availability
Discovery deadlines will greatly influence the review strategy. For instance, a DOJ second request with a 30-day turnaround might be handled differently that a standard run-of-the-mill litigation matter with a more casual discovery schedule. Accordingly, there are different strategies which are determined by the nature of the request and the resources of your organization.
Large law firms may have the resources to handle the demand for resources to setup a review and staff reviewers. These resources include networking staff, security, desk top support, training, staffing managers, litigation support staff, and human resources. Smaller firms might rely on third-party hosting vendors and temporary staffing agencies to provide staff. (Internal capabilities may also drive the review platform and format. See Selecting a Platform for Review.)
Third-party hosting vendors also have resources issues. There are large "blue chip" EDD processing / hosting vendors and there are small "local" vendors with varying degrees of competence. One might rely on a large vendor for a sizable project with a tight deadline yet expect to pay a premium price. Compare to a smaller vendor who might be more aggressive in pricing while struggling to find the resources to complete the project within the deadline. (See Selecting a Vendor.) One should also consider the production format, as the time to print paper or convert to TIFF needs to be built into the review/production schedule.
Previously we discussed the need for staffing to do the review and if the process was going to be handled in house or contracted to a third-party hosting company. (See Managing the Review Process.) Either way there are areas the internal staff must consider and manage:
- Technical staff for connectivity and performance. Does the firm have a strong IT / Litigation Support organization? Can a large team of reviewers be adequately supported during a round-the-clock / weekend review?
- Experts in the data being reviewed. Did the data collected get loaded correctly to the review platform? Who will answer questions about the data should there be a processing error or password-protected file?
- Data management. Which custodian's data should be prioritized? Who assigns review data to the reviewers? Who will decide search criteria for keyword culling or privilege grouping?
- Contract attorneys v. internal resources. Does firm have enough internal reviewers?
- Performance. Will the review be complete by the deadline? Are the reviewers going fast enough and meeting their quotas?
- Quality Control. Someone in the internal team needs to spot check the work being done by the reviewers. Are the privileged document tags consistent? Are non-responsive documents truly non-responsive?
Client Cost Considerations
The processing and review of electronic evidence can potentially amount to the highest percentage of litigation costs. Accordingly, it is critical to consider various review and processing options and the overall impact on the litigation budget.
In-House v. Outsourcing
Many firms now have the tools and expertise to handle moderate amounts of EDD processing (de-duplication and conversion to review platform). While internal processing may be a lower-cost option, the requirements of the case must be considered. If the litigation support staff becomes overwhelmed, the entire review could get sidetracked. The other major cost consideration is staffing the review. Internal attorneys will cost more than outsourced temp attorneys. However, the quality of the review may not be the same. Finally, what level attorney or paralegal will conduct the review? Junior-level associates may seem attractive based on lower billable rates, but the review might go faster and have a better quality if mid- to senior-level attorneys perform the review.
Should the project be outsourced to a vendor, there are many factors to consider which have a direct impact on the project costs. Primarily, how does the vendor charge? While the industry seems to be gravitating towards a "per gig" pricing model, some small- to mid-size vendors still charge per page. Be sure to understand the services included in the per gig/per page pricing. Is de-duplication included? Does the fee include hosting? Are there extra costs to produce numbered TIFF files after the review? Additionally, does the vendor have the internal expertise to consult with your team on ways to reduce costs?
Figuring out the processing costs based on per page/per gig pricing is relatively simple compared with estimating the billable time necessary for the review. As previously stated, this is the part of the litigation that will cost thousands, perhaps even millions of dollars. However, once the data has been processed and loaded to a review tool it becomes easier to calculate the costs, timeline and necessary resources based on simple math. For instance, if a reviewer can review 30 documents and hour and there are 50,000 documents, the review will take approximately 1670 hours. That would amount to 208 working days. If staffed with 10 reviewers, the review would take roughly 20 days and cost about $370,000 (1670 hrs X 220.00/hr).
Based on the above example, it is critical to stay on track with the project budget and timeline. If the review progresses slower than projected, it is best to identify this early so additional review resources can be added, or review criteria can be modified. Additionally, the clients cost expectations may need to be managed along the way - "we estimated the review cost based on 30 documents per hour but due to the complexity of the material, we are really only able to do 20. Therefore, we need to add reviewer and the costs will increase..."
New concept searching and grouping tools can be employed to "jump start" a review by allowing a reviewer to see "like" documents one after another. In fact, studies show that reviewing similar documents together can speed the review by 10x. Grouping the documents also allows the review to be prioritized. For instance, all documents likely to be privileged or everything likely to be non-responsive can potentially be assigned to individual reviewers based on their expertise. A brand new associate can be assigned "things likely to be garbage" while a more senior reviewer might tackle potentially privileged docs. This grouping and resource allocation can have a dramatic impact on review costs.
Regardless of the staffing, vendor or technology decisions, there is still a fundamental QC process that needs to occur. The goal of the review should be a consistent end result in which responsive, or privileged documents are correctly assigned. If the reviewers stray from their training and incorrectly "tag" items, significant amounts of time are necessary to correct the erroneous work product. Constant sampling and QC review along the way facilitates a "touch once" approach, which translates directly into efficiencies and cost savings.