Promised boring topic blog post in form of README snapshot.

The README with tables removed has a Flesch Readability Ease score of 48.5, slightly worse than the average license text. I did not try to write intelligibly, though I should. The topic may have subconsciously restrained parenthetical discursiveness.

Automated readability metrics for public copyright licenses. Give style a list of plain license texts, generate HTML table containing metrics.

In Debian, style is available in the diction package.

License texts are referenced from the SPDX licenses list. Other license curiosities are included in licenses-other.

sh license-readability-html-table.sh licenses-spdx/*.txt licenses-other/*.txt

Background

Part of one of the goals of the Creative Commons (CC) licenses version 4.0 effort is to make the licenses "readily understood". One way to test that is with automated readability metrics, on which CC licenses version 3.0 score poorly (previous versions scored much better). I checked an early version 4.0 draft, and scored much better, more or less back to version 2.5 scores, quite an accomplishment given it is a more sophisticated license in many ways. I did not check again until the near-final 4th draft was published. Its score is not as good as early drafts, probably to be expected as details were settled, but still a big improvement over 3.0. I intended to blog the early 4.0 draft improvement at that time but didn’t get around to it.

In the meantime I’d peeked at the readability metrics for various free/open source software licenses, in part to see if copyleft-next scored better than comparable licenses (probably, though comparability is problematic). With the CC 4.0 licenses nearly final, I started a blog post about readability of various licenses, and ended up with this README and associated files.

See Caveats and Output below for readability metrics for about 228 licenses. There probably will not be any big surprises awaiting anyone familiar with the usual relatively popular licenses. A small selection of licenses not in the SPDX licenses list (including CC 4.0 drafts and copyleft-next versions) are at the end.

Next

Drafters understandably try hardest to "get the legal details right". But if "licenses are the constitutions of software communities"12, even a little bit (I think a casual reading of that quote makes licenses far more central than they are, or implies impoverished communities, but will take its repetition as an indicator of licenses’ social importance), perhaps yet more effort ought be put into making licenses more understandable.

• There is probably a large literature on readability and understandability of contracts, legislation, regulation, and other legal texts, which ought be digested for lessons for the public copyright licensing community. Apparently many jurisdictions have "plain language" requirements for contracts. Some U.S. states require insurance forms to have a minimum Flesch Reading Ease score. Is this an indicator that readability metrics are useless, or should free/libre/open/software/knowledge communities be embarrassed that they have failed to self-regulate to this level?
• Cloze testing and subjective evaluation (both requiring humans) and natural language processing/machine learning based metrics are suggested by a readability tools site in addition to simple automated readability metrics. The site, by Michael Curtotti, is presumably discussed in his forthcoming paper The Right to Access Implies a Right to Know: An Open Online Research Platform for Assessing the Readability of Law. Could some of these tools be useful for evaluating licenses? Barriers would include lack of interest needed to pay for human testing, and a relatively small corpus of license texts. Hopefully the source code for this platform will be made available.
• Attempts to increase readability and understandability outside of changing the words in a license text could be evaluated, including summaries, FAQs, choosers, and typography and other design elements around web publication of the license text itself.
• There are many additional obscure licenses intended for "content", "data", "government", and "hardware designs" not included in the SPDX license list that could be analyzed.
• Non-English license texts could be analyzed with language-appropriate metrics. In addition to the few CECILL licenses included in the SPDX licenses list, targets could include the many official language versions of EUPL versions, unofficial translations of GPL versions, License Art Libre, various public sector-focused licenses, and hundreds of CC license "jurisdiction ports".
• To what extent is understanding of licenses social, gained via hearsay, not based on reading license texts at all? If social learning currently predominates, does this indicate that license readability and understandability are unimportant? Or that their lack constitutes an obscurantist barrier to participation by people not socially connected to existing communities, and increase other risks, such as non-compliance through ignorance, and being ignored by policymakers?
• Would it be valuable to use readability metrics to test other texts important to free/libre/open communities, e.g., documentation, codes of conduct, contributor agreements?

Caveats

General, with respect to the metrics:

• Metric explanations are available in the style man page. All are problematic.
• Lower numbers indicate better readability for all metrics except Flesch.
• None of the metrics incorporate text length, so correlations with character count ought indicate that longer texts tend to use more or less readable language. But 3 of the metrics positively correlate readability with longer texts, and 4 negatively, which might indicate no overall correlation (taking the numbers at face value, with no further validation).
• Not sure why Coleman-Liau’s correlations with other metrics are much weaker than among others; at a glance the formula is measuring the same types of things.
• Arbitrarily choosing to focus on Flesch, as it seems widely used, and its more is better makes for an easier combination with text length, "Chars/(Flesch>=1)", to indicate how painful reading an entire license might be.
• Flesch can be negative, so a minimum value of 1 is used for the pain calculation. This is arbitrary too.

The following tables are calculated in scratch.ods.

With respect to particular licenses:

• The CECILL licenses, except 1.1, are in French. These readability metrics may not be tuned for French, though the results do not look weird.
• The CC by-nc-sa-4.0-drafts are drafts. Every other license analyzed is "released".
• GPL-[version]-with-[exception name]-exception are not complete licenses, should be appended to the relevant GPL-[version]. However, standalone (as provided by the SPDX licenses list) provides an idea of how readable each exception is.
• LGPL-3.0[+] incorporates GPL-3.0 by reference, so it is not directly comparable to GPL-with-exceptions above, nor with other licenses.
• Some licenses (most notably [A]GPL and FDL) have a preamble or addendum which explain the license’s purpose and how to use the license. This makes such a license longer, but arguably increases understandability in a way not captured by an automated readability metric.
• The only license with a negative Flesch score is the Historic Permission Notice and Disclaimer (HPND), which is deprecated. It deserves the score, basically being a template with many optional and fill-in parts.
• The longest and also most "painful" to read license, the Adaptive Public License (APL), is also basically a template with options and fill-in parts.
• The shortest and also least "painful" to read license, the Fair License might require too much imagination about what "usage" means to actually be easily understandable.

Output

License

Original files in this project are disjunctively licensed under all licenses in the SPDX licenses list 1.19 (215) and those included in licenses-other (13). Take your pick of any or all.

License texts purport to be under various terms; see each individual license text.