Measuring legal expertise with KUJI Ratings

The stages through which professional expertise develops is something we’ve known about for a long time. The craft guilds of medieval Europe ranked craftsmen according to their abilities as novices, apprentices, journeymen, experts and masters.

For some vocations it was (and still is) relatively easy to measure expertise. We can, for instance, identify an expert bricklayer, chef or artist by the quality of their finished products.

The professions are more difficult because results and expertise are not always correlated. A lost case doesn’t necessarily mean your lawyers were no good, and a win won’t necessarily mean they’re the best. There are just too many variables.

 

The KUJI Breakthrough

So how do we accurately measure the expertise of lawyers? The key is understanding that expertise lawyers are expert thinkers.

In the case of lawyers who are true experts in their specialties, the ground-breaking research of Meisterline’s Founder and CEO, Dr Peter Macmillan, has shown how these lawyers consistently solve complex legal problems with degrees of efficiency and accuracy not found in their less capable colleagues.

These critical cognitive skills – and more – are captured by the KUJI® acronym:

Knowledge – The technical domain knowledge of the relevant area of law.

Understanding – Being able to comprehend the scope of the problem with an informed and empathetic appreciation of how it affects the client.

Judgment – The ability to assess a non-routine situation, give well-considered advice, and effectively self-monitor intuitive insights.

Intuition – An immediate response, tempered by judgment, based on a high-quality solution without second-guessing or undue deliberative doubt.

The last two Judgment and Intuition measures – the so-called “wisdom” components of expertise – are consistently linked to lawyers’ higher levels of proficiency as legal specialists.

 

KUJI Rating Numbers

KUJI Rating Numbers range from 0.1 to 9.9 and quantify a lawyer’s level of expertise in a specific area of law.  Each number represents a specific point along a continuum of cognitive development.

KUJI Bands describe in qualitative terms the key developmental milestones based on a lawyer’s KUJI Rating. There are six KUJI Bands corresponding to the traditional stages of expertise development, namely, Novice, Developing, Competent, Proficient, Expert and Master.

 

 

Frequently Asked Questions

  • How can I find out more about the research behind KUJI Ratings?

    The best place to start is the seminal book written by our Founder and CEO, Dr Peter Macmillan, titled Unlocking the Secrets of Legal Genius: Measuring Specialist Legal Expertise Through Think Aloud Verbal Protocol Analysis. This book provides a comprehensive review of the expertise systems used in Medieval craft guilds in Europe, the breakthroughs made by cognitive scientists in the 20th Century, and Dr Macmillan’s own landmark doctoral research which involved one of the most extensive studies ever undertaken examining the thinking skills of expert lawyers.

  • Can you tell me how KUJI Ratings are calculated?

    Unfortunately, we can only do so in broad terms. In addition to the proprietary nature of our methodologies, we do not reveal the details of how we calculate KUJI Ratings to ensure that no lawyers have an unfair information advantage over other lawyers, and to guard against the gaming of our systems.

  • How do you guard against lawyers gaming your ratings?
    Guarding against the gaming of our ratings is one of our highest priorities. The integrity of our ratings is critical for both the lawyers we assess and those relying on our results. Accordingly, we strictly limit access to our methodologies and assessment parameters. We also embed within our processes various anti-gaming safeguards, some of which are obvious and others much more difficult to detect. Should we suspect that a lawyer or law firm is attempting to manipulate our ratings – which will very quickly become apparent to us – we may refuse to release any ratings and in some circumstances report those involved to the relevant regulatory bodies. We may also refuse to provide any future ratings assessments.
  • How long does it take for a lawyer to receive their KUJI Rating?

    Ratings are conducted in batches, usually on a firm by firm basis. Ratings will generally not be provided until the last lawyer has completed their assessment questionnaire. There is then a tabulation period and time for preparing of our Discovery Document, which covers the analysis of ratings and related metrics as well as individual profiles for each lawyer. We normally allow a window of two weeks for all eligible lawyers to complete their questionnaires, though larger firms with international offices can take longer. Then another week for our own follow up work to be completed. We will then arrange a time to report back to the firm’s senior management.

  • Do you test in-house legal teams?

    Yes, we do. But generally only larger in-house teams where lawyers work as specialist attorneys.

  • What do KUJI Ratings actually measure?
    There is a short answer and a long answer. The short answer is more appropriate in this FAQ section. The longer answer is best delivered in person, usually during our meetings and seminars. That said, a lot of information can be gleaned from this website and our various explanatory documents and White Papers. 
     
    KUJI Ratings measure the following critical cognitive skills – and much more – as reflected in the KUJI acronym:
     

    Knowledge – The technical domain knowledge of the relevant area of law.

    Understanding – Being able to comprehend the scope of the problem with an informed and empathic appreciation of how it affects the client.

    Judgment – The ability to assess a non-routine situation, give well-considered advice, and effectively self-monitor intuitive insights.

    Intuition – An immediate response, tempered by judgment, based on being able to arrive quickly at a high-quality solution without second-guessing or undue deliberative doubt.

    The last two ‘Judgment’ and ‘Intuition’ measures – the so-called ‘wisdom’ components of expertise – are consistently linked to lawyers’ higher levels of proficiency as legal specialists.

    Our focus on ​these particular elements of expertise, reflects the reality that lawyers who do not develop these critical competencies will be, at the end of the day, unable to meet their clients’ legal needs, regardless of how responsive, empathetic or personable they may be.

    Further information is provided on this website and in our various explanatory documents and White Papers. We also provide presentations, workshops and seminars, where we can go into much greater depth on our measurement process.

  • Do you assess a lawyer's responsiveness, empathy skills, personability and other so-called "soft skills"?

    KUJI Ratings are not designed to measure a lawyer’s emotional intelligence, interpersonal skills, personality type or business development ability. We do know, however, that these factors contribute to the cognitive development of expertise – and training in these areas can accelerate an attorney’s development as a legal specialist.

  • What does "KUJI" mean?

    It’s an acronym of Knowledge, Understanding, Judgment and Intuition.  

  • How much does it cost for a lawyer or law firm to get their KUJI Ratings?

    Contact us so we can send you an Information Pack, which includes our pricing schedule.

  • What is a Discovery Document?

    A Discovery Document is the report we prepare for the senior management of a law firm or client organisation. It’s made up of several parts, some of which are free while others must be purchased. Essentially, the Discovery Document reports our assessment results, the KUJI Ratings for the lawyers we’ve assessed and, where applicable, benchmarking data comparing different law firms and legal teams. The Discovery Document also provides detailed cognitive profiles for individual lawyers and a range of qualitative and quantitative data.

  • How often do lawyers need to assessed and their ratings reviewed?

    KUJI Ratings need to be reassessed on an annual basis to ensure reliability. This is because expertise levels change over time – usually increasing – making annual assessments a necessity. 

  • How do I know my assessment data are securely stored?

    Data security is built into our assessment platform. We started by finding the right data centre to host our servers, to ensure that your data is secure and never leaves Australia. Even the DDoS protection we use utilises local scrubbing to ensure data is not routed overseas. This Tier IV Data Centre is certified to ISO 27001 and 27018 security and privacy standards. It has also been independently IRAP assessed for Government classified data, and with a SCEC rating for physical security.  In terms of our own internal controls, see our Privacy Policy.