The Digital Squeeze

The Digital Squeeze Episode 7: Benetech

Aug 17, 2020

EBSCO Product Director Kara Kroes Li and E-Book Product Manager Rob Smith talk to Benetech Director of Content Partnerships, Michael Johnson, about e-book accessibility, why accessibility is a spectrum, the GCA certified publisher program, and more.

[Music]

Julie Twomey:

Hello, and thank you for listening to The Digital Squeeze podcast. This is your host, Julie Twomey.

In this episode, we are discussing the newly established partnership between EBSCO and Benetech. Benetech is a non-profit organization that offers a variety of different programs around social justice and global literacy. Their globally certified accessible program works directly with publishers and analyzes their e-book workflow, and offers guidance on how to adjust their workflow to develop a fully accessible e-book. I'm also joined again by two of my colleagues from EBSCO Information Services, from our ebooks team. Our guests will discuss the importance of e-book accessibility, and the value Benetech brings to publishers in the library community.

I am excited to welcome three guests to the program. First, we are joined by Michael Johnson, the director of content partnerships at Benetech, a non-profit organization that uses technology for the social good. Welcome to the program, Michael.

Michael Johnson:

Thanks for having me, I appreciate it.

Julie Twomey:

Our next two guests are colleagues of mine from EBSCO Information Services, representing our eBooks division. First, I'd like to welcome back Kara Kroes Li, the director of product management for e-books. It's great to have you back on the podcast, Kara.

Kara Kroes Li:

Thanks for having me, Julie. Happy to be here.

Julie Twomey:

Great. And we also have Rob Smith, an EBSCO platform product manager, representing our e-book accessibility. Thank you for joining us again, Rob.

Rob Smith:

Thanks. Glad to be here.

Julie Twomey:

In this episode, we are focusing on Benetech and their publisher certification program. Benetech offers GCA certification to publishers to verify that their ebooks have an appropriate level of accessibility. Benetech and EBSCO are also excited to announce our partnership, dedication to libraries, collaboration on upcoming projects, and commitment to evaluating publishers and promoting e-books that are fully accessible to all users. So, let's get the conversation started.

Before we get into what Benetech does and the benefits that it offers publishers, libraries, and researchers, I wanted to discuss the topic of e-book accessibility and why it is so important right now. Rob, when folks hear that an e-book is not accessible or usable by all people, regardless of disability, what does that mean and who does it really affect?

Rob Smith:

Well, there are many reasons why someone wouldn't be able to read a standard print or an e-book. The one area that I think people think about immediately may be people who are blind, but there's really a spectrum of ability. Many people have low vision, color blindness, or some other vision impairment. And, there are also people with motor disabilities that can make manipulating the book a challenge. That could be a permanent condition like paralysis or multiple sclerosis, or even a temporary disability like a broken arm or hand. Finally, you have people with cognitive impairments, like dyslexia or ADHD, that may make reading a challenge.

E-books, in particular, have an inherent advantage over print, for the cases I just mentioned, in that they're so much more flexible and can be more easily used with assistive technology. In some cases, in ways that weren't really possible until recently. But, e-books still need to be created with accessibility in mind to take advantage of this. A poorly accessible e-book is a wasted opportunity to allow people to consume content conveniently, not to mention with more independence and dignity.

Julie Twomey:

Absolutely. Michael, did you want to add to that?

Michael Johnson:

Yeah, sure. That was a great answer, Rob.

One area of disability I think we should discuss is dyslexia, because if you can't decode the words, then obviously you can't read them. Properly formatted accessible ebooks really help with people who are struggling with dyslexia, and that's not an insignificant group. Actually, it's the largest statistical group that we talk about, as it relates to print disabilities. There are more people with dyslexia than there are people who are left handed, so this is an enormous group of people. And they are really helped in the decoding process, and in the reading process, when it comes to accessible e-books.

Then, there's also elements inside the book themselves, which present tremendous difficulties. Images without proper descriptions, or charts or graphs, anything like that which would require somebody to have "normal vision" to be able to see. An accessible e-book helps make those items inside the book not only accessible from the point of, they know what they are, but also it fits it in with the surrounding context and aides in learning.

Julie Twomey:

Right. Michael, obviously with the current pandemic, the need and usage of e-books is high, and making e-book accessibility a really hot topic. Can you also describe some other reasons why accessibility is at the forefront right now?

Michael Johnson:

Sure. So for about 20 years, we've been talking about this movement from print to E. I've always said it's print plus E, because there's lots of reasons why people would still want a print book, even if the print book is in Braille for the blind.

But, what's happening now with COVID, it's driving more and more content to digital. You also have the idea that there are more and more legislation, not just the United States, but around the world, that are about inclusivity. Add to that, of course, the current social justice activity happening here in the States, around being more diverse and inclusive. Those are all the educational, and social justice, and practical reasons.

Then, of course, there's a lot of settled case law around accessibility and inaccessibility. I just want to talk for a second about that. People view accessibility as some sort of tax, or some sort of thing which is done for a specific group of people, and they miss the incredible benefit that everyone gets from other accessibility issues. For instance, you go some place that has a television on but no sound, but you've got closed captioning. Okay, so we can all see what's going on on the television, and that's because of an accessibility issue. Another one would be curb cuts, or ramps instead of stairs, these were things which were mandated by Federal law for accessibility purposes, but then have come to benefit everybody. And, let's not forget the various speakers: Siri, and Dot, and Echo, and those things, again, were originally thought up to help people who had certain disabilities, and now are used by everybody.

Rob Smith:

Yeah. Actually, a recent one that I saw that was really interesting was, with the pandemic going on and so many people wearing masks, people have been relying more on voice transcription to get things across. That's something that really originated to help out with people with deafness.

Michael Johnson:

Yes, that's excellent.

One of the things specific to COVID is, here at Benetech, we've had over 120% increase in new membership this month this year versus this month last year, every month, since March. So it's clear that COVID-19 is having a massive impact on people who need accessible content, because they're coming to us and registering for our services.

Julie Twomey:

Wow. That really speaks to the need. I think it's important for us to talk about the levels of accessibility, so that people can understand that there's a variance here.

So Kara, can you describe why there is a variance? And, what it means for an e-book to be born accessible, or fully accessible?

Kara Kroes Li:

So we've been trying to express to the community that accessibility is a spectrum. It's not a yes or no, is or is not accessible thing. There can actually be a huge difference in a user's experience of two resources that are both considered "accessible."

So first, it depends on both the e-book itself, and the delivery platform or the website being accessible. So if the platform or the website's accessible, then a user with a screen reader or a keyboard can search for the e-book and get to it. But, if the e-book is just text, like a textual PDF for example, without tagging that allows the user to navigate, or without descriptions of images, it's going to take that user a lot longer to get to what they need out of that resource, if they can find what they need at all.

For example, Michael mentioned tables and charts. If they're looking for information that's contained in a table, and that table isn't tagged so that a screen reader can interpret it intelligibly, it'll be really difficult for the user to understand that data. That experience could be considered accessible, because the user was able to navigate to the resource on an accessible platform, and the file contains text that was readable by the screen reader. But, that's not at all the experience that we, in the library community, want for our users.

So on the other hand, if the publisher has made an EPUB file that's fully accessible as part of their publishing process, they've followed all the guidelines for semantic markup, for tagging, for descriptive or alternative text within that file. So the user can navigate through the book based on headings, they can find the places in the book they're interested in, they can interpret tables and charts. And, if they get to an image, they'll know not just that they've encountered an image, but that they'll understand the meaning behind that image, why the author included it, so the user experience is totally different.

What we're striving for is that their experience is as efficient as possible, and is as equitable as possible, to that of their sighted peers. That's what we mean when we're talking about fully accessible, or born accessible books. If an e- book is produced as fully accessible right from the get-go, and not marked up later insufficiently, then we call it born accessible.

Rob, did you have something you wanted to add?

Rob Smith:

Sure. Yeah, one other thing that I think is helpful to point out is the idea of 100% compliant can be a bit of a myth. Very few websites or content are completely free from web content accessibility guidelines violations, particularly the more complicated a website gets. That's just the same as there are typos or software defects in a website or a book. But, that doesn't mean striving towards full accessibility is not a good goal. We all have an obligation to get as close to meeting these standards as possible, and to convey this is a transparent way. And, when we find issues, to fix them as quickly as possible.

There's a lot of work to do on this front, across this industry. We need to make sure we're considering the user, and not just striving for bare minimum compliance.

Julie Twomey:

I totally agree, Rob. From what I've heard, it really sounds like it starts with the publisher. So Kara, what motivation does a publisher have to make their content accessible?

Kara Kroes Li:

Well, we think that publishers have an ethical obligation, or at least incentive, to create accessible e-books because we believe that people have a right to information. It's important for their academic success, for their health, for their prosperity, et cetera. But, we also believe that it will be good for business. It opens up more readers, more buyers for content. And, specific to the library community, we hope that librarians with an interest in accessibility, and inclusivity, and social justice will favor publishers that are making an investment in accessibility, and thus prove that it is good for business.

So we're beginning to recognize and highlight publishers that are taking steps toward accessibility, and investing in it. We hope to prove that hypothesis, that it is going to be advantageous for them financially because we recognize that it does cost money for publishers, to make titles fully accessible.

Julie Twomey:

Right, right. Since we work so closely with publishers, does EBSCO have a way to automatically check the accessibility of publisher files, and offer that publisher feedback?

Kara Kroes Li:

Yes we do. We have actually built an internal validation tool. We've leveraged publicly available things like EPUB Check, et cetera. But, we run all of our EPUBs through this check, we've run more than one million titles now, to date. We do an automated check for WCAG AA Compliance. That gives us great information. It tells us that the publisher has met certain standards, and at least is aware of the standards. But, there are a lot of accessibility features that an automated check just cannot access.

For instance, the quality of the descriptive text for images. You just need a human to inspect that file in great detail to really know that the title has reached its potential. The only way to really verify at scale that titles are fully accessible is something like the Benetech certification. Where they're certifying the workflow, and they're ensuring that the publisher knows how to do these things, they've figured out how to get it into their workflow. That's why we're so excited about the Benetech certification.

Julie Twomey:

Michael, did you want to comment on that?

Michael Johnson:

Yeah. There are a bunch of tools that are readily available, whether you're a publisher trying to understand what to do, or whether you're a librarian who's trying to understand which publishers are doing born-accessible books. So there are tools out there from the DAISY Consortium. There's a website called Inclusive Publishing.org, which goes through and actually ranks the various reading and system software available. There are many conventions which focus on accessible content. AHG, which is Accessing Higher Ground. There's a publisher face-off every time that conference comes up of a number of different organizations, including Benetech, take files from random publishers and just score them, out there in front of everybody. Or, at least reveal the scores right in front of everybody.

Publishers can check those resources, librarians can check to see who's doing what, as far as it relates to reading software. And you can tap other libraries to see what they're up to. Southern New Hampshire University's a great example, or the University of Virginia, or Northern Arizona University, these are all places that are doing an excellent job with their accessibility programs.

And of course, they can always come to Benetech and find out more about both accessibility, and about our program and what publishers we're working with.

Julie Twomey:

That's great, and I think that segues lovely into what to learn about Benetech. Can you give our audience some information about how it got started, and the services that you provide to publishers and the library community?

Michael Johnson:

Sure. Benetech is a charity, a non-profit, which got started about 20 years ago. Our founder wanted to use some of the wealth, basically, that he had acquired through years on technical projects. He founded Benetech ... it's a mashup, basically. So it's technology for the benefit of the social good, so that's where Benetech the name comes from, so it's about 20 years.

We have a number of different programs around social justice. Our largest program, by far, is Global Literacy. Global Literacy does a number of different things, as it relates to individuals with print disability. Bookshare, which is a 900,000 record strong repository of accessible books, which are available for free to people who have certified print disabilities. We also are the undergirding technology to a number of other national libraries, RNIB in the UK, the Library for the Blind in Ireland, Vision Australia, as well as a corresponding library in Canada. So that's another place where we're serving end users, and library users, too. And then there's the GCA program, Globally Certified Accessible.

The GCA works directly with the publishers to teach them how to make sure the output of their create workflow is accessible. Once they go through that process, and they become a GCA Certified Publisher, then from that point forward, the books that they create, there's a stamp in a field inside the EPUB file that shows that they're certified, and that were certified by Benetech. Hundreds of titles are already available, and over time, it will be thousands, and tens of thousands, especially with help from organizations like EBSCO, to make these titles available for purchase by an institution, or by an end user.

The goal to all of this, our little internal mantra is "every book for every user, every day." Much like Rob said, we don't think we will ever get there, for a variety of reasons, not the least of which is some titles are just too far in the back list for publishers to go back and claim them. And then, of course, there will be publishers who won't participate, for whatever their motivations are. So that was a long answer to your question, sorry. But, that's what we're up to, as it relates to literacy and helping educational institutions, libraries, and readers.

Julie Twomey:

I think that's tremendously helpful, Michael, and not too long at all.

Since you mentioned the publishers, could you let us know which publishers have achieved Benetech certifications, and which ones are working towards it?

Michael Johnson:

Sure. We have two publishers that have already been through and are fully certified, that's MacMillan Learning and Guilford Publications. And then, in the very close, but we don't predict timelines because it is a lot of work that we've been talking about, we've got Wiley, Simon & Schuster, University of Michigan, McGraw-Hill, Kogan Page. We even did a project for the World Bank. They're not a traditional publisher, but we went through and analyzed a bunch of files for them. So that's what's going on, basically, up to the minute when I checked the dashboard this morning.

Julie Twomey:

That's great. Like you said at the beginning of the program, your requests are going to just increase. So, the time is now for publishers.

How can publishers get started on the road to certification with Benetech?

Michael Johnson:

The first steps are things they can do for no money, and inside their own operations. There are two very simple digital tools available, I believe each of them are part of EBSCO's service, but a publisher could do it on their own.

That's a tool called EPUB Checker. What EPUB Checker's going to do is it's going to look at the publishers EPUB file, just to make sure that it doesn't violate any of the technical standards, so it's just formatting in the EPUB. Then, there's a tool called Ace, from an organization called DAISY, and Ace will go through and do automated accessibility checks. So it's going to look for things which can be coded, and things which can be determined by a machine, as coded correctly.

But, as Kara mentioned, a machine can't know if the description that you put in there of a portrait of Thomas Jefferson, or the Champs d'Elysee, or whatever, that your description is accurate, so there are a series of manual checks. Publishers might want to go through those. If they don't have the expertise, I would caution against that.

Then, of course, the most important step about certification is to go through the certification program. We've streamlined it, it's super easy. There's three levels for the publishers: simple, moderate, and complex. We don't go title, by title, by title, by title for every title they ever create. We're working with them on their workflow and it's an iterative process. They send in a title, we send back their score. We don't just say, "You failed," we say, "You failed, and here's why, and here's what you should have done." And then, Benetech staff are available for follow-up conference calls, so it really is a cooperative process. We help the publishers get through, so that their workflow consistently creates born accessible, and GCA certified titles.

And then, on an annual basis, we do some random spot checks. There's two reasons for the random spot checks. One, we don't know what's been going on in the publishing house over the past 365 days. Did they change conversion vendors, did they change some kind of process in editorial, have they changed their markup policies? So we need to check to make sure they're still following the rules and regulations. And also, because these are international and open standards, they're changing. So everybody in the GCA program gets a quarterly note from us that explains all the newest aspects of our testing, and all the newest aspects of any standard that we follow.

Julie Twomey:

Wow. That just feels like a must-have for publishers. So to Kara, what is EBSCO doing to recognize or highlight Benetech certified publishers?

Kara Kroes Li:

So right now, we've created a website where we are highlighting publishers that have partnered with Benetech. There's still only a few publishers out of our 1500 or so that are certified, and certification takes a while. But, we wanted to get the word out as soon as possible, that these are the publishers that are taking those first steps. We want to acknowledge them, and recognize them for the effort, and for that investment. We're hoping that giving visibility to them within the library community will give libraries the option to, as I said before, favor those publishers, and support them, and to help them try to recoup that investment.

Julie Twomey:

EBSCO has officially entered into a formal collaboration and partnership with Benetech. Can you describe why this partnership is important for EBSCO and libraries, and what folks can expect in the future?

Kara Kroes Li:

We stand to learn a lot from an organization like Benetech. They have developed so much expertise about publishing workflows, and end user needs. We see the benefits to end users, of fully accessible EPUBs, but we also, as an aggregator, see the reality of where the publishing community is right now.

We want to do everything we can to educate and support our publishers on their accessibility journey. And when publishers tell us they don't know where to get started, we're happy to share with them that Benetech exists as a charity, how they will help them get started. It's not a financial relationship, this is purely for the benefit of the library and publishing community.

Rob Smith:

We really hope that this allows us to use our position in the industry to positively influence publishers to make strides on accessibility.

Kara Kroes Li:

In the future, as the number of certified titles increases, we intend to bring that Benetech Global Certified Accessible badge into the acquisition process in GOBI, at the title level, so that libraries can be informed about the accessibility of the e-books that they're buying.

Rob Smith:

We're helpful, as Kara had said before, that this will be a way for publishers to understand that accessibility can positively impact the bottom line.

Julie Twomey:

And conversely, why was partnering with EBSCO an important step for Benetech?

Michael Johnson:

There are a number of reasons for this. Certainly, EBSCO is a major player in the library marketplace. They've been very aggressive and very supportive of all matters of accessibility. One thing that we absolutely must do as a non-profit is to find business partners who can help amplify and coordinate our message. That was a critical part.

Another part is EBSCO showed a real understanding about the issues around accessibility. What of their own systems are not accessible, what of their own data storage techniques need to be altered to handle all this accessibility information. They've been very open with us about how we could work together.

Once every book in the world is already accessible, that'll be fantastic. Even already fully accessible, that'll be fantastic. But there's still great deal of work that goes on between when the author submits a manuscript and the publisher publishes the book. And then, from that stage, between when the publisher releases a book to the sales and distribution wholesale channel, whatever it is, and the reader has a successful experience, EBSCO has a lot of those systems involved. So the partnership there, to build out the accessible EPUB ecosystem, was very important to us.

As it happened, our first partner campus, which was Southern New Hampshire University, is a big EBSCO customer. So, that all worked out very nicely, and the three of us have co-presented several times. It's been a great partnership. There's simply things that we couldn't do, as a non-profit, that EBSCO is making possible.

Julie Twomey:

That's great. We can hear your passion for libraries in your voice, which brings me to the fact that you have a very unique connection to libraries. Can you just let our audience know a little bit about your background?

Michael Johnson:

So I always start with the bad joke that I'm new, it's just 35 years for me. Actually, in October it'll be 35 years, straight up, in the library world.

I started programming on the mainframe, dealing with Federal library tapes, and OCLC regional tapes, and microfiche, and laser discs, and all that stuff. All the way down from the mainframe, and the minis, and the VAX, and down to the PC. And then, back up again, to web based content through the years.

I was a 20 plus year member of the American Library Association. My wife was an outpost librarian, before we moved out here to the Midwest. My oldest son was a university library director, and now is a director of a public library. My youngest son is in library school right now, as we speak. My middle son used to work for a small independent publisher. I was on the MARC advisory board for about eight years. Yeah, I have a real library card, and if this was a videocast, I could show you my actual library card which I take with me everywhere I go. Yeah, I'm a library guy.

Julie Twomey:

I love it. I love my library card, I still have mine, too. It just brings to the forefront that you know libraries, and you know what their challenges are. So when it comes to accessibility, how can libraries strengthen their knowledge and train their staff?

Michael Johnson:

There's some resources there as well. So daisy.org is a great resource to understand about accessibility. DAISY as an organization gives almost weekly webinars about accessibility, about what to do on campus, what to do from a training standpoint, formatting details, create details. It's a great webinar series, all that stuff is absolutely free. Inclusive Publishing.org is another site to, to get information about what systems are or are not compliant, and why they are, and what they're lacking in. You can always go to the Benetech website.

To be honest, you can just email me, michaelj@benetech.org, I'm happy to take those. I'm completely serious about that. As well as look to some of the leaders in the library space about this. I mentioned Southern New Hampshire University, they're doing great work. The University of Virginia is doing great work. The Charleston Conference, which I believe is going to go all virtual this year, we had a great presentation last year about the whole EPUB ecosystem. And EBSCO was actually part of that, now that I think of it.

There are a number of resources available. But, I'm 100% serious, if anybody in the audience wants to email me, Michael J, like Johnson, @benetech.org.

Julie Twomey:

Thank you, Michael. I know from our end that we're really doing all we can to support libraries at this time. So Kara, how is EBSCO helping libraries to improve accessibility for e-books?

Kara Kroes Li:

So EBSCO's doing everything that we can to be transparent about the accessibility of our products. Librarians are very much dependent on their vendors and on their publishers for that content, so we have expanded our focus to the publishers themselves.

When I talked about evaluating their content with the automated checks, we provide detailed reporting to the publishers of how they're doing on accessibility, where they're falling down, giving them even title level metrics, and directing them to resources like Benetech where possible, and suggesting areas of improvement. So, we're definitely focusing on the publisher side.

And then, as I talked about before, we want to help them make decisions based on accessibility information. It's important for libraries procuring content to know where they're choosing accessible resources and where they're not.

Michael, do you want to add to that?

Michael Johnson:

Yes. This is a great area.

One of the things that librarians have, far greater than anybody else in this conversation has, and that's the ability to control purchasing. If the librarians step up, and they ask the publishers, or demand to the publishers, that they want accessible content, that will be the fastest road for all of us. That'll increase the amount of content available, the amount of publishers who are participating in the program, these would be better end user experiences for all of your users, not just those with print disabilities. And, it also addresses a major legal obligation on campus, to provide content which is accessible for all students at the same time. That's one thing.

As far as working with EBSCO and the publishing community, I think we have the four best pieces here. We have a huge provider of library services, we have the library taken care of. You have EBSCO, who's an intermediary between the publishers and the library. And then, you have us a Benetech here, who's trying to help everybody with the accessibility issues, so that all works very well.

I just can't drive home the point strongly enough that the more libraries request of their publishers to create accessible content, the more good will be done for all of us.

Julie Twomey:

How can libraries and users offer feedback and recommendations to the EBSCO eBooks team about accessibility?

Rob Smith:

So we have an address, accessibility@EBSCO.com, that several of us monitor including myself. We're very happy to take questions there. We truly love to get feedback on accessibility. There are some serious accessibility nerds here at EBSCO, and it's great when we're able to take in-practice use cases and communicate them back to the teams that build our products. Sometimes it challenges our assumptions, which is a lot of fun.

Kara Kroes Li:

Yeah, I would agree. I would just add that we love feedback, both from users, or about users, and their struggles or successes. But, we also like feedback from librarians on what we're doing and on these initiatives, like this one with Benetech. It helps us to know whether what we're doing is useful, and whether there are other things we could be doing, or other organizations we could be working with, to help improve the accessibility of the ecosystem and help librarians achieve their goals. Really, don't hesitate to reach out to us. All of it, user feedback or librarian feedback, can go to us directly or to accessibility@EBSCO.com. Rob and I will see those messages.

Julie Twomey:

I know our teams love feedback, thank you for that.

Michael, I just wanted to address COVID-19 again, as we wrap up the program. What, if any, changes are you making to your business model? And, has demand picked up, as you mentioned, due to an increase for the accessible resources?

Michael Johnson:

Yes, absolutely. The demand has been huge. As I mentioned, 120% or more increases, month on month registrations, every month since March. That's been an incredible uptick. COVID is not causing the people to need the content, they always needed the content. But, some of the in-person support, maybe they had a reader, an actual human who would read to them or some other large-scale assisted technology machinery available, people used to do those things that way. Well, now that you can't go anywhere, your only outlet is digital content which is fully accessible, so we're seeing a huge uptick.

As far as changing business models, as a charity we don't really have a business model in the traditional sense of things. But, we have certainly deployed any and all resources we have available to us in increasing the collection size, which has gone from 800,000 to 900,000 just these last few months, which has been fantastic. And also, increasing user support, because all of those new users, whatever percentage of them, are going to need some additional help beyond that. That's where our focus has been, as well as doing things like this, finding partners both in the assistive technology side, and in the vendor community side. Those are the major changes we're making. We've seen no let down, even in summer. We have a huge summer reading program going on. COVID's really made a big impact on our numbers.

Julie Twomey:

Can you just remind everyone what the best course of action is if they want to contact you?

Michael Johnson:

Sure. You can always go to the Benetech website and find information and apply to those. But, anybody should feel completely free, around any of the issues we discussed today, or anything to do with accessible e-book content, just reach out directly to me, Michaelj@benetech.org.

Julie Twomey:

Thank you so much. And I want to thank everyone for joining us, and sharing your insight today. I want to ask our listeners to please go to our website to learn more about EBSCO eBooks, and that's at www.ebsco.com/products/ebooks. Thank you for listening, and take care.

[Music]

Transcripts are generated using a combination of speech recognition software and human transcribers, and may contain errors.