Creating Digital SAT Tests with College Board's Question Library

You read it right. A week back, we just completed designing four mock Digital SATs. Without any doubt, we declare they are the most reliable, most authentic, and most accurate mock Digital SATs ever created directly from the official CollegeBoard's Educator Library.

And what’s more, we’re giving them away for free. Yes, free. 

Before the College Board Educator Question Bank was only useable for individual question use or sectional tests, we worked very hard on creating them. We sweated on the smallest details. We banged our heads against challenges at every step. But we achieved it. And we got the perfect mock SAT.

But then we recalled the seatbelt story from Volvo. 

In 1959, Volvo engineers developed the three-point seatbelts. Today, they save about a million lives every year. The seatbelts were so revolutionary Volvo could have earned a ton of money from its royalty. 

But guess what? 

Volvo left the patent open so that every automobile manufacturer could use it, without paying a single cent to Volvo for royalty. Volvo did it as a part of their commitment to creating safer cars, because they cared for people.

So as part of our commitment to help the entire student and tutoring fraternity, we’re giving away the mock SATs absolutely free. 

The most reliable mock Digital SATs? 

Everyone claims they have the most reliable, the most authentic mock SAT, right? We don’t blame them. After all, many tutors work hard to deliver their best. They are well-intentioned.

But creating a mock SAT that accurately mimics the actual Digital SAT is more than just good intentions. A well-designed practice SAT is a good idea, but how do you know it is identical to the actual SAT?

The answer is: you can’t.

Only the College Board can give you an authentic mock test. Because they’re the ones that create the actual SATs. Only if you use the actual SAT questions can you create genuine mocks. 

So we did precisely that. We used actual SAT questions published in the Educator Question Bank. 

Is that all? Of course not. 

Anyone could do that. Even a second-grader can pull out 150 questions randomly from the Educator Question Bank and say she’s created a true mock. Nope. That’s not how it works.

In that case, what did we do so that we’re able to say these are the most realistic mock SATs ever? 

By the time you finish reading, you’ll agree we’re right. 

Where did we start

As you probably know, you can view and download the actual SAT questions from the College Board's official website. Just google “SAT Educator question bank” and you’ll find them too.

The site asks you to choose the assessment you’re looking for. Choose SAT and you’ll see a drop-down menu, as shown below.

Now you see two sections: Reading and Writing, and Math.

When you select Reading and Writing, you’ll see 4 domains under this section. Below is what the screen looks like (we selected all the types; you may choose differently):

That’s how we downloaded all the questions from the Reading and Writing section.

Similarly, we downloaded all the Math questions, which is how we got all the original SAT questions. 

Collecting all the questions

The Educator Question Bank has 1,079 actual SAT questions for the Reading and Writing section. And for Math, there are 938 actual SAT questions. So in all, we started with 2,017 (1,079 + 938) questions that were earlier asked in the actual SATs.

Out of these 2,017 questions, we found that 570 questions were already covered in the Official SATs. That left 1,447 questions from which to create mock SATs.

While you see only 98 questions in any SAT each Digital SAT contains 147 questions (more on that in a minute). A quick calculation told us that out of the 1,447 questions, we could create at least 9 mock SATs (1,447 divided by 147). 

Sounds simple, right? 

A little analysis told us we won’t be able to make 9 mock SATs. Not even 8. We’d be able to make only 3 mock SATs from these 1,447 questions. 

But first, let’s see why each mock SAT needs 147 questions (and not 98).

Every digital SAT needs 147 questions

Surely you remember how the Digital SAT is different from a paper-based SAT. 

A paper-based SAT follows a fixed pattern. No matter how you perform on the first or the 15th question, the test doesn’t change. But the digital SAT works differently.

The digital SAT is adaptive. We’ll quote what we wrote in our earlier blog: An adaptive test is different; it adapts to the performance of the test-taker. What questions you’ll see next depends upon your performance in the earlier questions. In that sense, an adaptive test is dynamic.”

Here’s how the SAT adapts to your performance. Let’s say you and your friend Robin both start with Module 1 of the Reading and Writing section, and you did very well in this section. 

The test algorithm understands you have performed well, so in Module 2, it will show you questions of slightly higher difficulty. Against that, Robin didn’t perform very well in Module 1. The test algo notices this too, so in Module 2, Robin will see questions of slightly lower difficulty. 

Effectively, you and Robin see the same questions in Module 1, but entirely different questions in Module 2. Naturally, your score would be different too, since you were given difficult questions in Module 2. To be ready for this, the Digital SAT needs two sets of questions for Module 2.

The below diagram summarizes this.

So now you know why the SAT needs 147 questions (Feel free to brag about this in front of your friends, in case they don’t understand the rationale)

Why 1,447 questions wouldn’t give us 9 mock SATs

As we said, we found we wouldn’t be able to make 9 mock SATs out of the 1,447 actual SAT questions. 

To understand this, we’ll have to dig deeper. Merely looking at the total number of questions wouldn’t even begin to explain.

As we mentioned, the Reading and Writing section has four domains. 

• Information and Ideas
• Craft and Structure
• Expression of Ideas, and
• Standard English Conventions

Even after that, each domain tests different kinds of skills. For instance, the Domain ‘Information and Ideas’ can evaluate skills like:

• Inferences
• Central Ideas and Details, and
• Command of Evidence

Hold on, we’re not done yet.

Each question could belong to one of the three levels of difficulty 

• Easy 
• Medium, and 
• Hard

In the below screenshot, you’ll see the first two questions belong to the same domain Information, and Ideas, but their difficulty levels are different. So they cannot be used interchangeably. 

Similarly, the third and fourth questions have the same difficulty level, but the skills they evaluate are different (Central Ideas and Details vs Inferences). So one can’t be used in place of another.

For instance, in the screenshot, the questions with ID 87aa7bab and ID ed314256, both test the same skill of Central Ideas and Details, but their difficulty levels are different. This means you cannot use the question with ID 87aa7bab where the question with ID ed314256 was used, because they are of different levels of difficulty.

So, if you want to create exact replicas of the actual SAT, you need to match all parameters. For every single question.

If you want to faithfully mimic an actual SAT, your mock SAT questions must match:

• The number of questions under each domain
• The number of questions under each skill, and
• The difficulty level of these questions

Replicating the official SATs

Let's say the first 5 questions on the official Test comes with following domain, skill and difficulty:

If we want to create a Mock SAT like the official ones, we need to remain strictly comply to it's order of domain, skill and difficulty configuration. Even a slight change would make our Mock SAT less authentic, less reliable. 

Let’s change just one question:

Actual SAT

Mock SAT

Changing the difficulty level of just one question would make the Mock SAT less reliable. That means our Mock SAT would have to remain faithful to the original SAT across all the 147 questions.

We begin mimicking the actual SAT

Ok, so the next step should have been easy: go to the Bluebook app and find the questions with the configurations you want (domain, skills assessed, and difficulty level), and then create a mock SAT.

But there’s a catch. 

The Bluebook app doesn’t give you any of these details. It only allows you to take Official SATs for practice.

Those details are available in the College Board Educator Question Bank.

So we need the help of both Bluebook Official SATs and the question from the Educator Question Bank. 

As said earlier, we had 1,447 original SAT questions that were not covered in the Official SATs that the Bluebook app lets you access.

Here’s how we can manually create a mock SAT.

First, look at Question 1 in the Bluebook app. Here’s the screenshot:

But we don’t know the domain or difficulty level of this question. So we went to the Educator Question Bank and ran a Search for this question (see screenshot below).

Once we found the question, we noted down its domain, skill assessed, and the difficulty level. 

Now we searched for another question that matched this exact configuration. But now we searched in the database of 1,497 questions. Because these questions weren’t a part of the Official SAT, we’d be able to use it as a question in our mock SAT.

Soon, we found one.

Now we have our first question for our mock SAT. This question is an official question, matches the entire configuration of the Official SAT, but has not been covered in the Bluebook app.

Then we looked at Question 2 of the Official SAT in Bluebook. We found the same question in the CB Educator Question Bank and noted its configuration. And from among the 1,447 questions, we searched for a question that’d match this exact configuration.

Once we found one, we added this as Question 2 of the mock SAT.

And that’s how we’d be able to build the entire mock SAT, right?

Not really. Something is missing. Something important.

Avoiding duplication of questions within mock SATs

The method we described above is fine when we’re explaining things. But it is unreliable - or rather inadequate.

Here’s why. 

Assume that Question 1 is from Craft and Structure (Domain), Words in Context (Skill), and is Easy (Difficulty level). The Question ID in the Educator Question Bank is 441e2b9e.

Assume now we’re looking for Question 11 which requires the exact configuration. If we’re not careful, we’ll again pick up Question ID 441e2b9e from the Educator Question Bank.

In other words, we run a big risk of repeating questions in the same (or different) SATs.

If we worked on things only manually, there’d always be room for human error which would repeat one or more questions. So we wrote a code (details given at the end).

The code would run through the 1,447 questions and find a question with a configuration we wanted. But importantly, the code would keep track of which all questions have already been selected. That way it will easily avoid duplication; once we used a question, it would never be used again.

We formatted out Mock SAT1 the following way:

So we started working on Mock SAT2 which was to be based on BB2. We repeated the above exercise in its entirety to select 147 questions for the Mock SAT2. 

Are any questions repeated from the Offical Tests?

We wanted to make sure each SAT was accurate and yet unique. 

While doing all this spadework, we had to be careful that we didn’t repeat any of the questions across two or more Mock SATs. Put differently, we wanted to make sure no question appeared in both Mock SAT1 and Mock SAT2 (or even all three mocks).

After we finished Mock SAT2, we repeated the exercise for Mock SAT3.

But when we were almost done working on the Mock SAT3, we found that we had only 146 unique questions that would match the exact configuration of BB3. That meant if we were to complete the Mock SAT3, we’d need to repeat one question that had already appeared in one of the earlier Mock SAT.

We were not sure if repeating any question would be right. So we went back to the actual SATs published by the College Board. And we realized that in the 3 SATs they’ve shared, in all 17 questions were repeated. 

That was a relief. If the actual SATs repeated 17 questions, it wouldn’t be unfair on our part to repeat just one question.

So there you are: three most reliable, most accurate, mock Digital SATs ever. Nothing can beat that. 

The number in each cell indicates how many unique questions are missing for that section. For instance, under the Adaptive Easy module of the Reading and Writing section of CB4, there are 2 unique questions missing. Similarly, under the Adaptive Hard module of the Math section of CB6, there are 4 unique questions missing (see image below).

You’ll see certain numbers in each cell. Also, some cells are colored. So what’s with the colors?

The cells colored yellow or red indicate at least one question is missing. Against that, a green-colored cell means no unique questions are missing for that configuration. For instance, under the Adaptive Hard module of the Math section of CB5, there are no missing questions (see image below). 

And why are none of the cells of CB1, CB2, and CB3 colored at all? That’s because there’s no question missing in the entire test, so we chose to not color their cells (We ignored coloring one cell of CB3 where there was 1 question missing, because the rest of the sections and modules for CB3 were all taken care of).

Here’s why they are the perfect mock Digital SATs

We want to make sure you know you’re about to take the most reliable mock Digital SATs that you can fully trust.

So we’ll very quickly recap how we put together these mock SATs.

1. We identified the domain, skill, and difficulty level of each question in actual SATs.
2. Next, we downloaded all actual SAT questions from the Bluebook app.
3. After that, we manually searched for and identified the domain, skill, and difficulty level of each question.
4. Finally, we build the mock SATs, matching the domain, skill, and difficulty level of each question with the actual SAT.

The remaining mock SATs

Now we were curious as to whether we could create the remaining Mock SATs (namely Mock SAT4, 5 and 6), because it was obvious we’d need to repeat a few questions.

So we went ahead and created Mock SAT4 (that mimicked BB4), Mock SAT5 (that mimicked BB3), and Mock SAT6 (that mimicked BB4).

We found that in Mock SAT4, we had to repeat 11 questions, in Mock SAT 5, we had to repeat 16 questions, and in Mock SAT6, we had to repeat 9 questions. (Made up numbers). 

So although we have the Mock SAT 4, 5 and 6 ready, we don’t think they are fully accurate or reliable for students. So we aren’t going to release them as full-length tests.Instead, we will use these questions as sectional tests.

The first three Mock SATs are of course perfect for students and educators alike. So we went ahead and fed them into our software for Mock SATs.

Now anyone - whether you are a student or an educator - can take these Mock SATs. All for free, no strings attached.

The code

We want to be completely transparent and give away everything for free with the mock Digital SATs. 

And when we say everything, we mean literally everything. Even the code we wrote.

Feel like having an explanation on ChatGPT? Here’s the link

https://chat.openai.com/share/d3140e0f-4086-4d9b-a409-c31a5d2fa414

In the code, we removed the tags of domain and modules. That’s because each of the 29 skills (listed below) are mapped back to exactly one domain and one module. 

Why take our 4 Mock SATs

We know that by now, you’re fully convinced that the mock SATs we put together are the most genuine mocks that can be created.

Al the same, here’s a recap of all the reasons why should take our mock Digital SATs:

• All questions taken from College Board’s Official Educator’s Library by removing  questions repeated on the 4 official testsBlue Book
• Each SAT is an exact replica of an actual SAT - down to the domain, skills, and difficulty level
• Fully adaptive, like the actual Digital SAT
• Actual SAT-like test interface and formatting 
• Accurately recreates the actual Digital SAT environment
• Gives the most accurate assessment of a student’s SAT likely performance, since it resembles the actual SATs to the maximum extent possible

We haven’t stopped with just selecting the right questions. We have gone to great lengths to replicate the SAT environment. For instance:

• Formatting the questions to the last detail. For example, we italicized author names just the way the actual SAT does.
• Using identical fonts. We’ve remained faithful to the font selection to reproduce the real SAT experience. 
• Incorporating high quality images. We ensured they not only load faster but are also optimized for multiple devices.
• Coding all formulas in mathjax r latex
• Tagging questions just like the College Board to leverage the same on post-test analytics and reporting for each attempt

Additional things we did to mirror the real, official SAT:

• Replicating the testing tools: timer, answer eliminator, annotator, desmos graphics calculator, reference sheet, and more
• Displaying the test on a full-screen mode
• Providing a cheat-free experience by logging time if you minimize the tab, or switch the tab

Not just the questions, we replicated the SAT score reporting too

While the selection of the right questions, with the right domains and skills, is critical to designing Mock SATs that mirror the actual SAT, there’s more to it.

What happens after a student takes a Mock SAT? How do the Mock SAT score a student’s performance? How does Edison calculate scaled scores?

Here’s a quick summary. 

We assign a difficulty value from 0 to 5, respectively for questions ranging from ‘No difficulty’ level to ‘Very hard’ level. For the raw scores, the student is awarded 1 point for every correct response, and 0 for every incorrect response (or for questions skipped). 

Next, we calculate the weighted mean score using the below formula:

The final stage is where we calculate the scaled score, keeping in mind that the official SAT scores lie between 200 and 800 for each section. 

We calculate the sectional scaled scores using the following formula:

‍

Check out our earlier blog for a detailed understanding on how EdisonOS score the SATs, 

Note on scaled score:

The College Board uses a proprietary algorithm to scale scores. This algorithm is not available publicly, so obviously we can’t use it. 

However, to understand how the algorithm might be working and what rationale might lie behind scaled scores, we carried out our own research. Based on the insights we got from our research, we have developed our own proprietary algorithm. The scores that you see at the end of our Mock SATs are scaled using our algorithm.

Based on our research and our experience, we can confidently say that the scaled scores students will see at the end of our Mock SATs mimic the scaled scores of the actual SATs very closely.

Insights from your Test Report

Ok, so the first challenge to overcome was getting the right questions. Extremely difficult, but we cracked it. 

The next was recreating the experience of an actual SAT. We did that too. 

But how can we help students improve their score?

Detailed analytics of their performance is what is the most important task at this stage. How many did you attempt? How many did you get right? How long did it take?

Figure 4 gives the Scaled score of a dummy student who we’ll call Pam.

That’s a great start, but we know that’s not enough.

Now we take a deep dive.

By default, our tests will have questions of three levels of difficulty: Easy, Medium, and Hard. If you want your SAT training to be more intensive, EdisonOS gives you the choice of adding two more types of questions: Very Easy and Very Hard.

The analytics will break down your performance across the three difficulty levels. The doughnut chart below shows the performance of Pam. The three figures are the three types of questions she attempted: Easy, Medium, and Hard. 

The red part stands for the questions Pam got wrong, while the green part stands for the questions Pam got right. 

Even a quick glance will tell you that Pam got only a few Easy questions wrong. As the difficulty level increased, the proportion of the questions she got wrong increased too.

Now we’ll analyze Pam’s performance section-wise. Figure 5 below shows how Pam did in the Reading and Writing section. Here, she got all the Easy questions right but faltered a little with the ones with Medium difficulty level. As for the Hard questions, she got more than a quarter of them wrong.

Also note that she hasn’t skipped any questions; those would have been shown in yellow color.

Figure 6 shows the distribution of the time that Pam took for each question. The bars in red indicate the questions she got wrong. Interestingly, there is at least one question where she spent a considerable amount of time and yet got it wrong. 

Let’s now look at the category of the questions that Pam attempted and identify where she needs to improve.

For instance, she got the Cross-text connection question wrong (3rd bar from top). Even with questions on Boundaries (6th bar from bottom) and Standard English Conventions (5th bar from bottom) are her weak spots - she gets more wrong than right. 

Against that, questions on Information and Ideas (5th bar front he top) seem to be her strong point - she’s got most of them right. 

Figure 8 is the next break-up. It shows you the difficulty level of the question, the kind of question asked, Pam’s answer, the correct answer, the time Pam spent there (in seconds), and whether she has bookmarked the question for review.

As you can see in Figure 9, such detailed analysis is available for both modules (Baseline and Adaptive).

Proctoring experience the entire activity of the test-taker is monitored: a clear log is maintained so the students know. We call it mild proctoring, because all the student behavior is monitored, including whether the student tried to switch screens.

With each question, there is a detailed explanation of the best way of solving the question.

Conceptualized by Deepak Murugaian and co-written with Mayank Batavia, we extend our thanks for their invaluable contributions to this blog.

Disclaimer: This service is free, doesn't require payment info, and is for educational purposes only. Content ownership is not claimed. SAT® is a registered trademark of the College Board, which is not affiliated with, does not endorse, and is not involved in the production of our product/service. Furthermore, the College Board does not endorse or sponsor EdisonOS, nor is it affiliated with EdisonOS in any capacity.