Creating a Defensible Process Through Search, Analytics and Metrics

Trying to successfully manage a large electronic discovery project is a balancing act. Lawyers and vendors are walking a tight rope trying to keep the cost of the review as low as practicable while simultaneously ensuring that virtually all non-responsive and privileged documents are not produced to opposing counsel. The use of search (keyword, Boolean or concept), analytics and metrics improves the ability of a reviewing firm to increase the accuracy of a review while reducing its cost. More importantly, utilizing such technology can aid in the creation of a defensible process and protect a firm and its client, helping to ensure that if a privileged document is inadvertently produced, privilege can be maintained and the errant documents returned.

Rule 502(b) and Rhodes Industries v. Building Materials Corp.

Rule 502(b) states that when privileged documents are inadvertently produced to the opposing counsel, attorney-client and work product privilege can be waived if the producing party fails to act reasonably under the circumstances. In Rhoads Industries v. Building Materials Corp. , 254 F.R.D. 216, 219 (E.D. Pa., November 14, 2008), the federal courts delve further into this 502 reasonableness standard: "(1) The reasonableness of the precautions taken to prevent inadvertent disclosure in view of the extent of the document production; (2) The number of inadvertent disclosures; (3) The extent of the disclosure."

The relevant factors in a 'reasonable steps' analysis of 502 include "the efforts taken to prevent disclosure, the promptness of the effort to correct the error, the scope of discovery, the extent of disclosure, the overriding issue of fairness, the nature of the documents reviewed and the time restraints for production." F.R.E. 502 Advisory Committee Notes. Rule 502 Advisory Committee notes also state, "a party that uses advanced analytical software applications and linguistic tools in screening for privilege and work product may be found to have taken 'reasonable steps' to prevent inadvertent disclosure." The proper utilization of search, analytics and metrics in a document review process dramatically improves the reasonableness of a firm's precautions. By utilizing technology that enables a granular look into quality control and statistical sampling methodologies, a firm can more easily demonstrate to a court that it took action to look at the work product of their staff and/or contract attorneys and show precisely what levels of corrections were made in this process. The methodologies chosen combined with this tracking technology can create a defensible process by showing how effective it was in preventing documents that could have, but were not, inadvertently produced.

Since its enactment on September 19, 2008, F.R.E. 502 has been applied only in a handful of district court cases. See Rhoads Industries v. Building Materials Corp. , 254 F.R.D. 216 (E.D.P.A., November 14, 2008); Relion v. Hydra Fuel Cell, CV-06-607-HU, 2008 U.S. Dist. LEXIS 98400 (D.Or., Dec. 4, 2008); Reckley v. Springfield, 3:05-cv-249, 2008 U.S. Dist. LEXIS 103663 (S.D Ohio, Dec. 12, 2008); Alcon v Apotex, 1:06-cv-1642-RLY-TAB, 2008 U.S. Dist. LEXIS 96630 (S.D. Indiana, November 26, 2008). These cases do not state whether the use of statistical sampling or quality control procedures help create a defensible process against waiver. Nonetheless, these cases, particularly Rhoads, indicate that the more advanced the review procedures, the more likely that process is to be defensible.

In Rhoads, plaintiff inadvertently produced 800 pages of attorney-client privileged documents in the course of electronic discovery and document review, thus the defendant argued for waiver. Rhoads at 218. Judge Baylson employed a two-step test to resolve this dispute. First, the Court considered whether or not inadvertent disclose had substantially complied with the factors outlined in F.R.E. 502(b)(1)(2)(3) and found that plaintiff had met that burden. Id. at 226. However, because the "reasonableness" of plaintiff's review procedures was still in dispute, the Court considered the five factors announced in Fidelity & Deposit Company of MD v. McCulloch,168 F.R.D. 516 (E.D. Pa. 1996), to determine whether privilege was waived (similar to the reasonable steps analysis of 502).

Judge Baylson painstakingly reviewed the facts surrounding plaintiff's review process. After considering a variety of facts in relation to the five Fidelity factors, the Court concluded that the most significant factor is that Rhoads failed to prepare for the segregation and review of privileged documents sufficiently far in advance of the inevitable production of a large volume of documents. Once this lawsuit seeking millions of dollars in damages was filed, Rhoads was under an obligation to put adequate resources to the task of preparing the documents. An understandable desire to minimize costs of litigation and to be frugal in spending a client's money cannot be an after-the-fact excuse for a failed screening of privileged documents, just as I refuse to use hindsight to criticize Rhoads for mistakes that were made but perhaps unforeseeable.

Lessons Learned

In sum, despite (a) using an expert consultant, (b) employing advanced keyword search technology and (c) manually reviewing some of the documents identified by those keyword searches, plaintiff was found to have been deficient in its review of the e-discovery materials at issue. In finding a deficient process, the Court's analysis hinges upon the failure of the plaintiff to allocate a sufficient amount of resources to their e-discovery process given the amount (multi-millions of dollars) in controversy.

If plaintiff had used a proven document review analytical or metrics tracking system, the court may not have found that the first three Fidelity factors favored the defendant. Plaintiff's main failure was that they failed to do a sufficient manual review of the documents identified by the electronic discovery program. Instead of reviewing all of the 76,000 emails pulled by the consultants, plaintiff only manually reviewed some of the 76,000. Incredibly, plaintiff found several privileged documents within the small subset they manually reviewed but did not take the time to then manually review the rest of the 76,000 emails. In short, plaintiff's manual review revealed that the keyword search was flawed but plaintiff chose not to follow up on this revelation. If plaintiff had properly used a document review efficiency tracking software or system, they would have manually reviewed each of the 76,000 emails at least once and done QC review on select portions to ensure the accuracy of that first review.

In my opinion, the use of search, analytics and metrics would create a defensible process provided that it is properly employed by the producing law firm. While there is no case that sits directly on all fours with this assertion, there is also no case in which a firm employed a more sophisticated review system and was found to have waived privilege via inadvertent disclosure.