eDiscovery Review: Managing the Review Process

Every stage of electronic discovery carries unique elements that ultimately may impact the resolution of a matter. The process of reviewing electronic data presents a variety of challenges, and the decisions attorneys make in managing the review process are critical to guiding the course of a lawsuit or government investigation. Decisions about staffing and training are instrumental to successfully completing the review project, and strict attention to cost control is essential not only to the project, but ultimately to maintaining a good client relationship. This section discusses some best practices that should be considered when tasked with managing an electronic document review.

Lead Attorney Selection

The first step is to designate a lead attorney to manage the electronic document review. Choosing an attorney to act as the project manager responsible for the day-to-day review process is critical to the success of the review. The old adage about too many cooks in the kitchen applies equally well to the management of such a project. Although critical decisions may be elevated to more senior attorneys, the project lead often will be called upon to make immediate decisions about relevance or privilege. Therefore, the lead attorney must be knowledgeable about the facts and issues of the matter, and should have experience with similar types of review projects.

In addition, the lead attorney must be comfortable with the technology to be used on the project. A lead attorney who would rather print everything for review will not be effective leading an electronic discovery project. It is important to choose someone who believes in the technology and who will champion its use. If the lead attorney is fully conversant with the functionality of the review application, s/he might even have a shot at getting the senior partners to utilize the tool.

It is important for the project lead to be involved in all of the initial strategic planning and decision-making surrounding the project so that s/he is well informed and knowledgeable about any issues that arose during the planning stage. S/he should be fully involved in the selection of the review platform. A critical mistake that can be made by litigation support or firm IT managers is to fail to involve the lead attorney in the review tool selection process. In the end, it will be the attorneys using the tool ten hours a day for weeks on end. If they are not comfortable with the tool, efficiency will suffer. That will impact deadlines and ultimately client costs. A clear understanding of the power of the technology being used will make the project lead a more effective manager.

The lead attorney must dedicate the time necessary to effectively manage the review and the technology. S/he must be willing to work closely with the vendor or consultant retained and the firm's litigation support and IT departments to resolve issues that may arise in the course of the project. S/he also should be able to draw on experience to ensure that all relevant information is captured during the review and that all other project requirements are considered and dealt with properly during the course of the project.

Ultimately, the project manager often is the key to successfully completing any electronic document review.

Review Team Selection

The selection of the review team will necessarily vary from project to project. A number of variables will determine the makeup of the review team, such as the volume of data, the time in which the review must be completed and the budget allotted to the review by the client. Additionally, logistical issues often will require consideration. Some firms are able to maintain standing review facilities for large projects. Others rely on outsourced facilities to be able to staff larger projects. Finally, the complexity of the matter may play a role in how a firm decides to staff an electronic document review.

Depending on any or all of these issues, a firm may choose to staff a project entirely with associates. If a firm does not have the internal resources to staff a project with associates, it may choose to retain contract attorneys to conduct the review. However a firm staffs a project, the practical issues discussed below will directly impact the course of the review.

Training

Training the review team probably is the most important component of a successful electronic document review. Coordinated, well-thought-out training of the team with regard to both the specifics of the matter and the technology being used is critical to ensuring efficiency and accuracy. Unless all members of the review team fully understand the expectations, the technology and the overall goals of the project, the review likely will be inefficient or even incomplete.

The lead attorney should conduct a team meeting early on to discuss case issues, explain in detail the information to be captured or coded during the review, and provide a brief overview of the technology that will be used. A coding manual that can be used as a reference tool by the team throughout the project should be distributed and reviewed at the meeting. This document must be updated throughout the course of the review as new issues arise or the requirements of the review shift.

Some of the information discussed at the initial meeting and covered in the manual might be:

  • Specifics of the document request
  • Facts or issues at issue in the matter
  • Codes being used to define issues in the matter
  • Privilege
  • Confidentiality

 

The review team also should be given guidance with regard to "hot" or "key" documents that may need to be identified, as well as documents requiring redaction based on privileges or confidentiality that may apply.

Technical training must be specific to the technology and tools being used, and should include detailed information about the interface design and functionality. The vendor or consultant retained often will provide a day or two of onsite training. If not, web-based training sessions can be conducted by provider of the review platform. If an in-house solution is to be used, the law firm should be prepared to provide adequate training on the use and functionality of that tool.

The reviewers should be trained on the workflow of the project, including:

  • Locating their assigned document sets
  • Coding/tagging documents
  • Elevating documents for further review by a more senior attorney
  • Annotating documents to convey key information
  • Highlighting important passages for attorneys reviewing documents "downstream"
  • Redacting documents
  • Expectations regarding review rates.

 

Details and examples should be included in the coding manual. While the vendor will be responsible for the technical training, the lead attorney should demonstrate the coding process by coding a handful of documents from an actual review set.

The lead attorney should use all of the tools that are available to make the review as efficient and effective as possible and make sure the review team is adequately trained to take advantage of these tools.

Review Environment, Communication and Technical Support

Depending on the needs of the case, a centralized location for the review may be set up. It is equally acceptable for the review team to work remotely or within their individual offices, although this may require a greater need for internal communication protocols. In any case, the review environment should be well lit, be free of distracting outside noise, and be positioned so that no unrelated foot traffic runs through the area.

The review environment also should include appropriate computer technology that allows the review to be completed with a minimum of technical distractions, such as slow connections or software glitches. In order to ensure that technology supports the workflow and does not distract from it, it is helpful if the lead attorney identifies and designates a lead technology support person early in the planning process. Additionally, vendor/consultant contact information for technology support should be obtained and disseminated to the review team.

The technology lead should be involved in the initial planning process, ensure that the hardware and software needed for the review are correctly installed, and that all systems have been tested before the review begins. In this age of increasingly managed computing environments at law firms, the technology lead should work to ensure during the planning process that the review team will be able to access the full functionality of the review tool from the firm's computing environment. Once these issues are addressed and the review has started, the technology lead should be readily available to address any hardware or software problems that may occur as the review progresses. The technology lead also may act as the primary liaison between the review team, the firm's IT department and the vendor/consultant if technology issues arise.

Even though the review will be conducted online, the reviewer still needs an adequate work space with appropriate desk area to spread out reference material, sample documents, etc. Reference materials all should be contained in a three-ring notebook to keep the information consolidated and allow for easy updates. Similarly, if a centralized location is used, a white board should be available in the room to allow for posting of new rules, items of interest and other hot topics to facilitate communication.

As reviewers become more familiar with the document set, each will begin to see emerging patterns in the collection that may affect the determination of relevance and privilege. It is important that knowledge gained by one reviewer be shared as quickly and easily as possible with the entire team. While this can be done via e-mail, instant messaging and with daily team meetings, an easy way to facilitate effective information sharing is to locate the entire team (or as much of it as possible) in a single work area. In that way, team members can freely discuss and review not only new insights into the document collection, but also unexpected problems that arise as the review progresses.

It is important to determine as early as possible whether after-hours technical support will be needed. The review team may need to work multiple shifts or extra hours in order to complete a review that has a short timeframe for completion. If this is the case, then the technical lead should establish a contact protocol that allows the review team to obtain support as quickly as possible after normal business hours. Depending upon the specific needs of the review team, this protocol might include a cell phone number to provide more immediate access to the project or technical lead. The specific limits of after-hour support should be established in writing and understood by all involved.

Productivity and Quality Control

Monitoring and measuring the productivity of the review process and the review team itself is another critical factor in assuring that the team is progressing in a timely and effective manner. Reporting requirements for which metrics are tracked and how they are tracked should be determined before the review begins. Useful metrics to track include:

  • The number of documents or pages each reviewer is completing per hour or per day
  • How many documents are being marked as privileged, relevant, and confidential and by which team member
  • Total pages or documents reviewed to date
  • Number of pages or documents left to review
  • Estimated number of pages or documents that can be completed in the time left considering the number of reviewers available and their rate of review to date.

 

Each of these items will enable the lead attorney to track the progress of the team and identify problems with workflow throughput early. Particular attention should be paid to reviewers who are moving through documents more quickly or slowly than the rest of team. This may indicate they are giving either too much or not enough attention to the factors involved in the review, and proper retraining can be provided.

The review platform should allow reviewers to mark a document for further review when they are unsure about how to code it. This ensures that reviewers are not forced to make an uninformed decision. They also should be able to easily revisit a document and change the coding in the event that it becomes clear that their original decision was not correct, although sometimes the review tool and/or workflow make this difficult.

The review platform should also be customizable enough to allow for quality-control restrictions that can be applied to families of documents such as threaded e-mails and attachments. For example, the protocol of the review might be that if any e-mail attachment is considered privileged, then that makes the whole family group (e-mail and attachments) privileged. If such protocols do exist they should be captured and executed automatically by the software to ensure consistency.

To guarantee uniform results from the review team and ensure that deadlines can be met, some form of quality-control monitoring and reporting will be needed throughout the entire review lifecycle.

Quality-control monitoring may include a second-level review of all or a percentage of the documents in the review collection. Second-level review is performed by more experienced attorneys who are very familiar with the matter under review. This typically is done with regard to documents coded as privileged. In many law firms, all documents initially deemed privileged are reviewed by at least one additional and highly experienced attorney, if not two.

In addition to or in lieu of a second-level review, the lead attorney may randomly review the team's coding to check for inconsistent application of the review protocol. Both of these steps provide for validation of coding decisions and can bring to light inconsistent coding patterns that then can be addressed with the entire review team. Both should be done on a daily basis in order to identify problems with the review process or with an individual reviewer's understanding of the process.

Discussing quality control provisions with your consultant/vendor is advised as they may be able to apply some automated procedures to ensure consistency or highlight areas of concern.

Cost Considerations

Electronic discovery costs can carry enormous consequences for clients, regardless of their size. A large publicly traded corporation may face negative investor reaction if such costs are large enough to become a reportable event on an SEC Form 10-Q or Form 10-K filing. Small start-up companies may face a loss of venture capitalist funding and risk the ability to continue business operations. The consequences for an individual facing an expensive electronic discovery project may include bankruptcy or, in the event of a criminal prosecution, the need to weigh personal finances against the potential for prison time.

Therefore, it is incumbent on counsel to maintain tight control of costs at every step of the discovery process, including the review phase. It also is appropriate for counsel to expect and demand that the electronic discovery vendors and consultants retained on behalf of the client play an active and responsible role in cost containment. In many cases, the vendor/consultant has the most immediate view into the progress and costs related to a project.

In one way or another, each of the areas discussed above with regard to managing the review process plays a role in containing costs. Thoughtful organization, training and project management are the keys to this goal.

During the course of the review, there are several practical steps the managing attorney can take to maintain control over costs:

  1. Work with the electronic discovery vendor/consultant to develop the most efficient workflow for the project. Often, the vendor/consultant will be able to counsel the review team on the most efficient approach based on the strengths or limitations of the review tool. Do not assume that the workflow that worked in your last case using a different tool will work the same way on the next case and a different review platform. Work with the vendor/consultant to develop the methodology that makes the most sense given the needs of each specific case.
  2. When selecting your review tool, keep in mind the goals of the project. It may be that your case does not require all the bells and whistles that are available on many tools. Pick the tool that best suits the needs of the case and provides the review team with the most comfortable and efficient method of completing their work. Reviewer comfort equals reviewer efficiency, and that equals cost savings for the client.
  3. Maximize the power of the review tool selected to organize the review in the most efficient manner possible. Where appropriate, take advantage of bulk tagging/coding features to categorize e-mail/e-files ("documents"). Utilize search engines to locate similar documents so that they can be reviewed together in context. If the tool has concept analysis or concept searching, consider using that functionality to group related concepts or issues for the review team. Place more difficult concepts or issues in front of the attorneys with the most experience and/or familiarity with the case. Similarly, place documents that are potentially privileged in front of attorneys with the most experience in that area. As we all know, one can ask five attorneys whether a given document is privileged and very likely come away with ten different answers. Ultimately, the organization of the review will have a direct impact on the efficiency with which the review team conducts its work, and that translates into cost savings for the client.
  4. Structure your review team with an eye towards efficiency, speed and accuracy. Although some matters require a literal army of contract attorneys just to get through the volume of data to be reviewed, that is not always the case. Whenever possible, staff your review with attorneys who are intimately familiar with the facts and issues involved in the litigation. By doing so, you may reduce the necessity of reviewing documents multiple times. Minimizing the number of times a document is reviewed prior to production translates into lower costs for the client. Although a certain amount of QC review always is necessary, the better the review team, the less of an impact QC will have on the final bill. Of course, proper orientation and training are critical no matter what the size of the review team.
  5. Set realistic goals for the review team. Establish a review rate that will allow the team to progress rapidly through the data set with accuracy and efficiency. If you do not have historical data from your own experience, work with the vendor or consultant to develop a plan. Setting unrealistic review rates only increases the chances of having to re-review data when it is determined that quality suffered by moving too fast.
  6. Maximize your use of the reporting tools available in the review application you choose. Many tools now provide tracking on the progress of the project as a whole as well as the individual reviewers. By ensuring that reviewers are meeting expected goals with regard to speed and accuracy, you will be better able to control costs. If the review tool selected does not have automated reporting capabilities, work with your vendor/consultant to develop a methodology for calculating progress.
  7. If the review is dynamic such that new data is being processed and uploaded into the review tool over time, or if productions are taking place on a rolling basis, establish a schedule with the vendor/consultant to provide periodic reports of costs to date. It also is important to obtain immediate notification of any unanticipated costs even before the work is done. It often can be the case that an unexpected cost will require rethinking the parameters of the project. One of the most important things to avoid is the surprise bill to the client.
  8. It almost goes without saying that the attorney managing the review must keep the responsible partner abreast of ongoing costs and, most importantly, the inevitable bumps in the road that impact the amount the client ultimately will pay for the project. Here too, avoiding the surprise bill is the key.

Managing the review in a thoughtful, efficient and coordinated manner, and working closely with your vendors/consultants, allows the firm to better control costs to the client.

Source: EDRM (edrm.net)