Category: Metrics

Client developmentClient relationshipsin-house counselInnovationInterviews

Review: Katrina Lee’s The Legal Career

511sXW1U++L._SX359_BO1,204,203,200_Katrina Lee’s new book on the business of law, The Legal Career: Knowing the Business, Thriving in Practice (West 2017), starts by exploring the design of a law-firm office. Lee points out that the law office can be seen as a microcosm of the legal industry: evolution, yes, but also persistent adherence to the old ways. Newer office designs place greater value on “flexibility, openness, and collaboration.” There is less of a differential between the size of junior associates’ office square footage and that of partners. Law libraries may look more like “a comfortable coffee shop,” or even (heaven forbid) be known as a “lounge-brary.” Less emphasis on space for physical books opens up more space for all employees. Despite these changes, some firms polish the walnut-grained panels the way things always have been.

The Legal Career goes on to chronicle law-firm billing conventions and salary structures, as well as the “precipitous” drop in solo practitioners’ salaries over the past 30 years, and a growing role for legal professionals who are not licensed attorneys. Lee cites research from Heidi Gardiner of Harvard that effective collaboration among law-firm offices and practices groups leads to increased revenue.

Lee now teaches at The Ohio State University Moritz College of Law; before that, she practiced law for 12 years including six as an associate and six as an equity partner. Her book brings together these two careers: it’s textbook for law-school courses in the business of law, with an admittedly heavy emphasis on civil practice. As Lee writes in her introduction, it is “law firm-centric.” It does include in-depth interviews with in-house counsel giving a helpful client’s perspective from within “the corporate law department.” Lee interviewed in-house counsel at a variety of companies ranging from Google to an Ohio-headquartered insurance company.

That is not to say The Legal Career is just a practical how-to manual for understanding the job market as it is and getting a job. That approach would simply replicate the current flaws and weaknesses of the legal industry; Lee’s book is more ambitious. For example, her interview with Dr. Silvia Hodges Silverstein delves into the Gender Billing study. Although female lawyers don’t work less than men and are not less productive, Dr. Silverstein’s study showed “clear” and “depressing” patterns: “[W]omen are assigned less strategic tasks, given more administrative work,” and “Male lawyers’ invoices were also less discounted than female lawyers’.”

The Legal Career explores other business problems and weaknesses such inefficiency and resistance to technological advances. Lee quotes D. Casey Flaherty: a client unhappy with a law firm’s advocacy or counsel should simply “get new lawyers.” But for complaints about the “content” and “production” of information as opposed to the underlying advocacy or counsel, a client may benefit from talking with their lawyer or law firm about better process and efficiency. In this regard, clients can drive change. Flaherty envisions the law firm as “long-term legal suppliers” and recommends more conversations between clients and lawyers to foster more efficient services for clients and more accurate, less discounted realization rates for firms.

Working efficiently raises the issue of incorporating project-management experts into the law-firm delivery model, and much broader involvement by professionals who are not licensed attorneys. Consistent with opinions of many in the law-firm innovation discussion, Lee questions the term “non-lawyer” as potentially “unproductive and unfriendly.” But what term should be used instead? And should lawyering be regulated differently to allow more “legal technicians” and the like? In this way, The Legal Career also takes on challenges with access to justice.

Near the end of The Legal Career, Lee explores the need for innovative legal education. Here again, the range of opinions offered is a strength of the book. One quote from William D. Henderson jumped out at me:

There’s a real opportunity here. Lawyers are always happy when they are solving their clients’ problems. It’s a great day when you solve your client’s problem. In this day and age, we’re going to solve a lot more problems better; that will bring a lot of psychic happiness to lawyering. The economic model for this is unclear, but it’ll sort itself out.

Lee doesn’t—and can’t—provide easy answers to such questions. She encourages creative discussion about the big issues facing lawyers, such as in a classroom setting. But a class on the business of law is not necessary to learn from this book. Anyone who reads The Legal Career will be challenged to reflect on their individual careers, the meaning and measurement of law-firm success, innovation in legal education and the legal industry, the role of lawyers in society, and the future of the profession.

 

 

 

 

 

 

 

 

 

 

Client relationshipsCross-cultural communicationEmotional intelligenceEmotional laborGender

Emotions in writing

Listening and speaking can be empathetic. Even reading (reading literary fiction, that is) is connected with empathy. But what about writing? And specifically, what about legal writing? The textbooks concur that writers are supposed to harness not only logos and ethos but also pathos in their appellate briefs and other persuasive writing. But what about the pathos—the emotion—in everyday legal writing?

Ever since learning about IBM’s Watson Tone Analyzer, I’ve wanted to try it on some legal writing. I wanted to find out what a “robot” like Watson has to say about the voice and emotions in contrasting legal-writing samples. Here’s what Watson can do:

The [Watson Tone Analyzer] service uses linguistic analysis to detect and interpret emotions, social tendencies, and language style cues found in text. Tones detected within the General Purpose Endpoint include joy, fear, sadness, anger, disgust, analytical, confident, tentative, openness, conscientiousness, extraversion, agreeableness, and emotional range.

As shown below, Watson offers an overall document-level analysis, and it highlights sentences that score particularly high on certain emotional indicators.

For this exploration, I chose the idea of an email sample because emails should be relatively short. Also, email is so prevalent in law practice. It’s a constant, quotidian part of life for many, many lawyers. Email doesn’t stop to ask, “Is this a good time to talk?” It just arrives. And it can have a major impact on the emotions of the recipient. “”When it comes to emails that are negative in tone, it makes you angry,” Professor Marcus Butts told Time Magazine, in an article about why email puts workers in a nasty mood—especially when checking email after normal business hours. The effect of such emails spills over: “Being angry takes a lot of focus and our resources and it keeps us from being engaged with other things.”

Given email’s potential emotional impact on the daily lives of lawyers, this post explores what the Watson Tone Analyzer had to say about two mocked-up emails. The two versions below both have the purpose of forwarding discovery requests to a client. The first version uses more formal language, and the second more conversational language. What does the Tone Analyzer say about these different versions? And in a more realistic situation, could the Tone Analyzer be useful to lawyers working on their communication skills? Following the text of the two emails, the post compares and contrasts how the Watson Tone Analyzer processed these emails.


Dear Ms. Smith,

Enclosed please find the Request for Production of Documents received yesterday (December 16, 2016) (“the Requests”) in the Acme v. Client matter. The Requests entail 136 different items, comprising five basic categories:

(1) foundational corporate documents for Acme;

(2) documents related to negotiation of the lease in question between Acme and Client;

(3) calendar items and email sent and received by three key employees during the negotiations, Jane Doe, John Smith, and Jamal Jones;

(4) documents related to discussions with Third Party Industrial; and(5) accounting records related to the lease between Acme and Client.

(5) accounting records related to the lease between Acme and Client.

After reviewing these Requests, please respond providing a convenient time next week for a phone conference to review them and discuss response strategies.

Sincerely,

Antoine Associate

Antoine J. Associate

Law Firm LLP

Citytown, RH

 


Dear Janel,

This message follows up on discovery in Acme v. Client. Yesterday we received another round of document requests. I’d like to set up a call with you next week to discuss them after you’ve had a chance to review them.

There were 136 individual requests, which are enclosed with this e-mail. The requests fall into five basic categories:

(1) foundational corporate documents for Acme

(2) documents related to the lease negotiation between Acme and Client

(3) calendar items and email sent and received by three key employees during the negotiations, Jane Doe, John Smith, and Jamal Jones

(4) documents related to discussions with Third Party Industrial

(5) accounting records related to the lease between Acme and Client.

Please take a look at the requests and then let me know when would be best for you to discuss them next week.

Many thanks,

Antoine

Antoine J. Associate

Law Firm LLP

Citytown, RH


So how did Watson analyze the emotions in these two messages?

Tone Analysis of First Sample:

The dominant emotion in this message was perceived as anger. Indications of disgust, fear, joy, and sadness were “unlikely.”

Screen Shot 2017-04-25 at 12.55.55 PM

The sentence-level analysis indicates that the anger emanates from plain, descriptive language (what the requests entail) and the final request (“please respond…”). The pink highlighted sentences below were flagged as moderately angry wording:

Screen Shot 2017-04-25 at 12.58.10 PM

The language in this message was viewed as both analytical and confident, but not tentative. The analytical content is highlighted here in blue, with the dark blue being more intensely analytical than the light blue:

Screen Shot 2017-04-25 at 12.59.25 PM

 

Interestingly, the confidence score appears to come solely from the signature block containing the words “Law Firm.” (The same is true of the second sample, where “Law Firm” were also the only text flagged for confidence. But the second sample’s overall confidence score at the document level is 0.00 (unlikely) compared with .63 (likely) for this first sample. More on that later.)
Screen Shot 2017-04-25 at 1.00.35 PM

The same text can be studied in more depth for its social tendencies including openness, conscientiousness, extraversion, agreeableness, and emotional range. For example, the language “Enclosed please find” was ranked as conscientious but not open, extraverted, or agreeable.  That language also scored high on emotional range. That same language was also flagged for showing anger.

Among the five items in the email’s numbered list of documents, item (3) seemed to be an emotional hot spot for Watson, scoring relatively high on all five of the emotional parameters. This result was notable because item (3) is the only item in the list that included individual people’s full names.

Here are the metrics for agreeableness, which form an interesting contrast with the second sample below. The greeting and sign-off are in light green, indicating moderate agreeableness. The only line with strong agreeableness was that same item (3) listing calendar items and emails sent by specific individuals by name. (In contrast, the second sample below tried to be friendlier and succeeded, as indicated by the more strongly agreeable opening and closing passages.)

Screen Shot 2017-04-25 at 1.05.01 PM

Tone Analysis of Second Sample

The second email was meant to be more friendly. What it accomplished, according to Watson, was slightly lessening the anger score and raising the joy score. The joy score is still “unlikely,” but it’s at .49 instead of 0.18 in the first sample. Although it’s less angry and more joyful, it also completely lost its confidence score.

Screen Shot 2017-04-25 at 1.05.46 PM

Despite the overall attempt to use friendlier language, anger still emanated from the email, specifically the sentence enclosing the discovery requests:

Screen Shot 2017-04-25 at 1.07.41 PM

But joy came from the revised beginning and closing words:

Screen Shot 2017-04-25 at 1.08.32 PMThe message did not rank on sadness, fearfulness, or disgust.

Watson’s evaluation of the language looks for analytical, confident, and tentative language. The more informal email’s language was also measured as analytical and confident, like the more formal first sample. Unlike the formal sample, it was also somewhat tentative. The source of this tentativeness was a sentence about what the writer “would like to do”:

Screen Shot 2017-04-25 at 1.10.19 PM

Not surprisingly, that same sentence was also ranked as agreeable:

Screen Shot 2017-04-25 at 1.11.12 PM

Quantitatively, the informal sample contained more agreeable language, ranking 0.89 on agreeableness compared to 0.67 for the first sample.

Conclusion

What did I conclude from analyzing these two samples using Watson’s Tone Analyzer? Like many AI analysis, it seemed to confirm what I think I already know.

  1. Legal information is not inherently happy, at least not in a litigation setting. The most “angry” language in both messages was the language simply describing the scope of discovery.
  2. Language that is more tentative and less confident may also be more agreeable. This correlation raises many questions: does tentative language compromise clarity? If so is it worth it to sound more agreeable? Different writers, readers, and situations will of course require different decisions.
  3. Watson’s Tone Analzyer may be helpful to some writers on a limited basis. As with any computer analysis of language such as Flesch-Kincaid readability scores, writers should ask whether the computer analysis could help them. I don’t see legal writers building Watson’s Tone Analyzer into a checklist for every email. But it could be a worthwhile exercise just on a couple of messages, to see what predominant tone Watson diagnoses.

And as with any computer analysis of language, take it with a grain of salt. I tested Watson on litigators’ favorite nastygram conclusion:

“Govern yourselves accordingly.”

The results are below but here’s a summary: Its predominant language was sadness (?????). Its most notable social tendencies, according to the Tone Analyzer, were extraversion and agreeableness.

Screen Shot 2017-04-25 at 1.28.57 PM

The “govern yourselves accordingly” analysis notwithstanding, a “robot” such as the Tone Analyzer could create an interesting exercise for trying different words and seeing how they measure. So . . . govern yourselves accordingly.

Note on use of Watson: these screen shots were taken on April 25 and 26, 2017. The metrics appear to have changed slightly from tests about six months earlier on identical language. Thus a final lesson is to know your tool and stay updated. Make sure you’re comparing apples to apples if relying on quantitative analysis of language. 

Legal communicationMetricsProfessional development

Better Lawyering Through Better Listening

What are your key strengths and weaknesses as a listener? How can you, as a lawyer or legal professional, actually become a better listener? What listening techniques can lawyers use to be more effective with clients, witnesses, judges and mediators, and others (perhaps loved ones)?

I’m pleased to be collaborating on a CLE workshop that will help attendees explore these questions and gain a better understanding of listening skills for lawyering.The workshop will take place in Tucson, Arizona, on March 10 in conjunction with the International Listening Association‘s annual convention.

Jennie Grau

Jennie Grau

Presenters Jennie Grau and Anita Dorczak will facilitate the session in person, and I am working on video contributions right now. One fun project has been working with several wonderful colleagues and students at my law school on video demonstrations of “bad” and “good” listening skills in legal settings. I must confess it was easier to write and create the bad examples. I’ve also been rounding up a variety of statistics related to listening and lawyering. (Blog post coming soon.) There’s a lot of great research out there, but also several widely accepted yet unsupported urban myths of listening.

Anita is a Canadian family lawyer and mediator, and Jennie is a communications consultant and speaker. Collaborating with these presenters has been a wonderful experience because they are not only smart and creative and talented speakers but also—not surprisingly—such skillful listeners. Arizona lawyers who attend the session are in for a unique experience including role-plays and the opportunity to take an individual “listening inventory.” Boring PowerPoint and lecture, this is not.

Anita Dorczak

Anita Dorczak

The International Listening Association is an eclectic group with connections to many disciplines such as health care, business, spirituality, cultural studies, teaching, and research.

Lawyers, mediators, and other legal professionals are invited to attend the conference for a one-day rate (4 hours of CLE). The full information can be found here. Fees for the session go to the International Listening Association. This is a labor of love for the presenters—which is actually the theme of the whole convention, Listening: A Labor of Love.

Emotional intelligenceGenderInnovationLeadershipLegal communication

Listening analytics?

One of my favorite sayings is from F. Scott Fitzgerald:

Slide1

Kenneth Grady’s Seytlines blog is an exercise in what Fitzgerald meant. In Grady’s essays on innovation in the legal industry—what it needs and where it is stagnating—human skills including “soft skills” have never been more valuable. Yet humans must use processes and systems and technology to avoid losing the competition to deliver value. Individual lawyers in all of their humanity have never been less expendable—or more.

Grady’s recent post Talking About Lawyer Performance illustrates the tension:

Providing legal services today involves much more than listening to a client’s problem and giving an opinion or delivering a document. It is a complex task in a fast moving environment that involves a much deeper and more nuanced understanding the environment in which the client operates. This isn’t an equation solely for large law firms and corporate legal departments, it is true throughout all levels of legal services delivery. Individuals’ lives are much more complicated today than 10, 20 or 30 years ago, so advising them isn’t as easy today as it was then.

This complexity manifests in the idea that legal-services delivery should be examined and broken into more distinct parts. This idea is pervasive throughout the legal-innovation conversation, and I’d like to think more about how it affects listening.

There may be a tradeoff in client satisfaction unless technological innovations are built with empathy and surgically precise understanding of how to approximate human interaction, and when actual real-time conversations and face time are crucial. On the other hand there will be a gain in client satisfaction if perceived unnecessary conversations where the client keenly feels the billing clock ticking are reduced or even eliminated. As I said, I’d like to think more about the delivery questions—and mostly I would just like to learn from those such as Grady and Patrick Lamb and Jeff Carr and others, the gurus in this area.

Beyond the questions of legal services delivery are deeper questions about what an individual lawyer does. (See Grady’s post on Defining the Unique Role of the Lawyer.) The analytical and problem-solving contributions are inextricably wrapped in the soft skills used to deliver them. As Grady has written elsewhere, “During the next decade, the skills that make up personality will play an increasingly important role.”

But do not believe that means the lawyer is unique beyond measure. Even the most human of human skills can benefit from systems analysis because even the most human of interactions can be measured:

Alex “Sandy” Pentland, who directs MIT’s Human Dynamics Laboratory and the MIT Media Lab Entrepreneurship Program, is one of the leaders in the people analytics field. His team developed sociometric devices—smartphones using special software—that teams of employees would wear during the day. The devices measured proximity to other employees, who was talking, engagement levels, and other data points. They did not capture what was being said. But, from this data Pentland’s team could determine which group dynamics led to more creativity or productivity. By altering the work situation, such as aligning work breaks rather than staggering them, Pentland’s team drove performance improvement along many metrics.

This was the part of the Lawyer Analytics post that really stood out. This blog has talked at various times about the problem of measuring listening. If you can’t measure it, you probably can’t assess it in a meaningful way. Perhaps these “sociometric devices” are the beginning of a solution to the problem.

When I first got started blogging here, I read a difficult but rewarding academic book, Talk and Social Theory: Ecologies of Speaking and Listening in Everyday Life, in which a scholar, Frederick Erickson, analyzed detailed transcripts of several conversations recorded in 1974: a blue-collar family at dinner, a college counselor and a student who was eligible for the Vietnam draft, a combined kindergarten-first grade class, and a medical resident and intern diagnosing a difficult case. He parsed every last detail of these conversations and even showed how they could be rendered with musical notation:

image

This book is where I learned the concept of the “conversation turn,” which essentially means taking over or handing back the conversational flow to your conversation partner. (See prior post on the “turn sharks” in law school.)

How do a bunch of random conversations in 1974 relate to legal skills today? Some things don’t change: Being a good listener means mastering conversation turns to keep the conversation going without taking over.  Just refer to Pam Woldow’s lengthy discussion of “manterruptions,” and the gender imbalance in who does the interrupting versus gets interrupted, to understand the relevance of conversation turns today. (Part I of Woldow’s series is here.)

The conversation studies in Erickson’s book were fascinating but clearly expensive to create and difficult to replicate.  With newer and more affordable technology like the sociometric device described in Lawyer Analytics, people won’t need to be invited to a scholarly study to get this kind of data. (To see the logical and alarming extension of these possibilities, read this article on “searchable speech.”)

The possibilities of these devices inevitably bring to mind FitBits. Ken Grady’s boss Stephen Poor has already covered that ground for lawyering generally in “FitBits, Data and Lawyers.” On quantifying communication specifically, it seems pretty likely that we will soon have relatively affordable “FitBits” for listening.

InnovationLaw firm managementLaw practiceMetrics

Listening and metrics of quality

If a client feels listened to, is that client likely to use the lawyer or law firm again? Maybe, but not if the lawyer listened deeply and sincerely while charging three times what the client expected for the work. Metrics for lawyers and firms get complicated very fast.

Courtesy Flickr/Scott Akerman/CC by SA 2.0

Courtesy Flickr/Scott Akerman/CC by SA 2.0

Kenneth Grady’s latest Seylines post points toward the lack of process (and process-based metrics) for delivering legal services. The lack of process makes it difficult to measure the services and compare them. Instead, “counseling and advisory skills” are viewed as what can and should be measured:

[M]any general counsel talk as if legal services delivered by one firm are not distinguishable in substance from those delivered by another firm. Rather, say general counsel, it is the counseling and advisory skills that separate the desirable outside lawyers from the rest of the pack. While soft skills are key qualities differentiating lawyers, until we become a process-oriented industry, legal services will not be interchangeable.

So I think what he is saying is, general counsel may be using metrics about soft skills because they don’t have “harder” metrics about process. Grady points with hope toward signs of better process:

As the ways in which lawyers handle matters become standardized, it becomes easier to compare what law firms do, the quality delivered, the value clients receive, and to find areas for improvement. This is the first major step to transforming legal service delivery from a world of inputs and outputs with a black box sitting between them, to a world of transparent legal services and costs.

I hope he’s not saying that “counseling and advisory skills” will become unimportant in a world of truly standardized legal services. I don’t think he’s really saying that, although perhaps he would like these skills to be measured in the background against a foreground with objective metrics of process.

Whether metrics for counseling and advisory skills are a good thing, or just a second-best waiting for something better, Grady’s post made me want to know more about the metrics for these skills. In particular: How do GCs measure whether lawyers and firms are listening to them? 

The most accessible list of metrics I found was published by the Valorem Law Group. (Thanks to Ron Dolin’s post on “Getting to New Law: Standardized Quality Metrics” for pointing me to the Valorem list.) I took a look at the common metrics suggested in the Valorem list to see whether listening was mentioned. It wasn’t explicitly, but it could play a role in quite a few of them.

Here’s a chart brainstorming how listening may play a role in lawyers’ and firms’ performance on a number of common metrics. The metrics are on the left; thoughts on listening are on the right:

Typical metric

How listening may play a role

Cycle time Effective listening could help resolve matters more quickly and reduce cycle time.
Performance to budget Effective listening can help counsel gauge how difficult a matter will be (e.g. reluctant or poor witnesses) and thus estimate budget realistically
Results to predicted outcomes Similarly, effective listening can help with more accurate predictions by teasing out bad facts and revealing problems with potential testimony.
Timely work completion Effective listening can help the lawyer understand the client’s preferences on setting up timelines (more flexible or more aggressive and strict).
Percentage of holdback awarded and buckets of holdbacks awarded These are incentives that are “indicative of widespread client satisfaction.” Effective listening could contribute to the overall effect of widespread satisfaction.
Re-engagement percentile and   re-use index A client is more likely to want to re-use and re-engage with a lawyer or law firm that listened to the client effectively. Or at least, a client is not likely to engage a firm or lawyer who didn’t listen.
Recommendation index A client is more likely to recommend a lawyer or law firm that made the client feel listened to.
Creativity index This metric “[r]equires client to assign a score on lawyer’s creativity in solving problems, structuring settlements, providing strategy ideas, etc.” Understanding the client’s goals and what the client can give up is an example of effective listening that contributes to problem solving. In general, effective listening enhances problem solving. (This claim is worth a more detailed post at a later time.)
After action ratio The Valorem Law Group post notes that an after-action review isn’t necessary in every case. Effective listening could help a lawyer gauge whether a client wants to spend time on this kind of review. Effective listening in an after-action session seems like it could be crucial to making the session productive, especially in a sensitive situation.
Quality of advice Effective listening could contribute by allowing the lawyer to have more complete information when crafting advice, and a better understanding of client preferences in receiving advice.
Quality of written product Listening indirectly contributes to good writing by giving the writer more information. “Listening” to how the writing sounds in draft form helps a writer modulate tone. Reading out loud and listening to the words can be very effective.
Quality of outcomes Listening can tease out weaknesses and strengths that the lawyer can then use to help the client understand what kind of outcomes to expect.
Wins v. losses Can listening contribute directly to wins and losses? I’ll make a case here: Poor listening can result in problems such as failure to make a record, so yes. Good listening can steer a lawyer toward the arguments that matter most to the judge, so yes. And effective listening can help a lawyer manage which cases are appropriate to go the distance as “contested proceedings,” thus affecting the overall set of cases measured in terms of winning and losing.
Collaboration As defined by Valorem Law Group, a collaboration event would be “two or more people meeting to discuss the case, brainstorm about strategy and tactics, and similar discussions yielding value for client.” Listening is crucial to these meetings. This one isn’t hard at all.
Transparency index Transparency seems to be more about what the lawyer says and shares than how the lawyer listens. But effective listening can help a lawyer recognize what the client wants in terms of frequency and manner of sharing, and when the client may not understand. (Giving a correct and detailed explanation that the client does not understand may not help a lawyer’s transparency rating.)
Total value index This index “[f]actor[s] in all metrics weighted in whatever manner client sees fit.” Listening could contribute to the client’s perception of “total value.”
Percent of claims resolved within 30/180 days of claim As noted earlier, effective listening could help resolve matters more quickly and reduce cycle time.

The list here does not include all of the metrics from the Valorem Law list. Although I try to relate almost every legal topic to law in some way, I just could not see the connection to metrics such as “effective disaggregation,” “average number of timekeepers per matter,” and “average seniority on team,” as well as a number of quantitative measures based on fees. (But wait. For average seniority on team, maybe there is a connection to listening. Ineffective communication with junior associates may lead them to seek opportunities elsewhere, reducing the pool of senior associates available for staffing.)

Another caveat is that not all legal matters involve the traditional definition of listening, as in some form of spoken and/or nonverbal input. For those that do involve meetings or phone calls at any step of the way, the metrics above suggest that listening can affect a pretty broad variety of metrics.

The Valorem Law post notes that many of these metrics are “subjective” such as whether a client would recommend a firm or lawyer and what the client feels is the “total value index.” And that brings us back to the difficulty of measuring listening in any context.