In our most recent release (Joule), we were excited to introduce our new Heretik Image Viewer. We built our Image Viewer with the reviewer’s workflow in mind, taking into account workflow pain points, specifically around coding discrepancies between native and extracted text views. Our product and engineering teams worked closely with Heretik power users to create a best-in-class review experience that allows for streamlined interaction on both the extracted text and the familiar native.
Head of Product Rishi Khullar and Senior Software Engineer Sam Miles both played crucial roles in designing and building out Heretik’s new Image Viewer. In this blog, Khullar dives deeper into the problem at hand, and Miles follows it up with an explanation of how our solution was built to solve it.
The User Problem
Before diving into the specifics of Heretik’s new Image Viewer, I wanted to provide some context to the problem we are solving. Please humor me by imagining this scenario:
You are a furniture store owner (congrats on the new biz!) and you have a commercial lease for your store space. There are ten or so data points that you want to pull from the lease just so you can have easy access to that information.
You are interested in capturing simple things like the term of the lease, how it can be terminated, and the specifics of rent payments. You want those data points in a spreadsheet for your records, so you don’t always to need to open up the super long lease agreement and read through it.
There is only one problem – your copy of the executed lease is a PDF, but it’s a scanned image (gasp). That means when you open it up in Adobe Acrobat, you can’t search it. You can’t select text and copy/paste into your Excel file. Forget about Control + F. You literally have to scroll through this giant beast of a lease and find what you want, then re-type it into the spreadsheet. You think to yourself, what is this? 1985?
This story touches on the problem at its core. However, the magnitude of pain scales with the number of contracts and the number of data points that you need to capture per contract.
Corporations can have hundreds or thousands of scanned image leases for which they need dozens of data points from each. This means that teams of attorneys need to split up batches of contracts, separately open them up, read through them, re-type information into spreadsheets, and then consolidate them all into one master file of structured contract data. If you’re reading this and this all sounds familiar, you know it’s a barbaric workflow that is wildly inefficient and error-prone.
So, what is the solution? In legal tech, we often jump to “AI” as the universal answer to all our problems. But our original furniture store owner just wanted a better way to navigate her scanned contract. She wasn’t looking for a robot to do the work for her, she was ready and willing to do it completely on her own. The frustration of pulling data out of a contract that you can’t search, navigate by clause, or even copy/paste text from is a user experience problem.
The Technical Challenge
We at Heretik strive to delight our users with the best possible review experience. Part of that vision is to provide the benefits of working with text (search, navigation by clause, auto extraction) while viewing the native document, even in the dreaded worst case of the scanned image PDF
Extracting text from images using Optical Character Recognition (OCR) tools is nothing new. The US Postal Service has been using it since the mid-1960s to sort mail. In a perfect world, you’d run OCR on scanned image documents, get perfect extracted text that looks exactly like the native, and could then simply review them in a text viewer. In reality, this process is error prone – characters are misread, page numbers are thrown in at random, and what is extracted starts to diverge from what is actually on the page. For this reason, reviews often require referring back to the native document to verify coding decisions.
To date, reviewers have been faced with a tough choice. Their first option was to use Heretik’s extracted text view with all its modern features and interface, but also have to verify coding decisions in the native view. Option two wasn’t any better – use the native and forego all the workflow efficiencies.
Our goal with our new Image Viewer is to give you your cake and let you eat it too. To introduce the ultimate review experience with an Image Viewer that doesn’t sacrifice quality of life.
From a reviewer’s perspective, you can now navigate realistic/familiar images the same way you would with extracted text. You can click sections or data points, view native documents with highlighting, section navigation, etc. All of this can be done while not disturbing or changing the original PDF.
Heretik’s new Image Viewer
How does this work? Other OCR services provide extracted text but toss out critical metadata about where a word was found on the native page. Using Heretik OCR, we’re able to capture this critical data. This allows us to create a detailed map of where each word is located in the native image.
Because we have this hOCR data AND we generate the extracted text, we are able provide a rich document review user experience.
For example, if our analysis extracts the ‘Confidentiality’ section from a contract, it tells us what character in the document the section starts and ends. With our hOCR data, we can take this start and end range to find what words are included and their coordinate location on the native image and then draw our highlighting appropriately.
Put it all together and you’ve now got the ability to do the following with Heretik’s new Image Viewer:
Oh and did we mention in the future, you’ll be able to send-to-field from the image?!
Interested in seeing Heretik’s new Viewer for yourself? Click below to set up a demo!
Rishi is the Director of Product at Heretik. Prior to Heretik, he worked as a product manager at Relativity. Rishi began his career as a lawyer and worked as a judicial clerk for the Virgin Islands.
Sam is a Software Engineer at Heretik. He is passionate about building elegant web applications to help people work smarter, not harder. Prior to Heretik, he was a software engineer at Thomson Reuters and TruePad.