Goal 1: Understand how my edtech product is being used

Are you interested in gaining new insights into edtech usage at your school or district? The implementation of products can vary greatly depending on usage and access. Being able to gain insights into the extent to which a product is being used and by whom is an important first step in understanding edtech effectiveness in your setting. Depending on when data is made available for your edtech products, you can run these reports monthly or every few months to understand how a particular edtech product is being used.

Goal 2: Understand how users are following recommended usage guidelines

Were users expected to engage with an edtech product in a particular way, and do you want to know if they followed those expectations? A product provider or education administrator might set expectations for users to use an edtech product (e.g., five hours per week). If you want to know how many users reached that recommended benchmark, this research goal is right for you. Similar to the first goal, these reports can be run monthly or every few months depending on how often data is made available.

Goal 3: Understand if greater use of my product relates to higher learning outcomes

Do you want to know whether increased use of an edtech product is related to improved learning outcomes? Do you want to know whether there are optimal levels of usage for your users? For this type of RCE report, you will only include users who received the edtech product or intervention (e.g., outcomes without a comparison group) to examine the relationships between edtech usage and an outcome measure (e.g., mid-year test scores). You will be able to examine how, if at all, learning changed over a period of time and whether usage relates to that change. This goal coincides with one of our most common RCE reports, and can be run as often as you have outcomes available.

Goal 4: Understand if students who use my product outperform other students

Do you want to know if there is a difference in outcomes between users who engage with an edtech product and those who do not (e.g., outcomes with a comparison group)? What about if there are optimal levels of usage for your users who used the edtech product? In this type of RCE report, you will include users who receive an edtech product or intervention (e.g., treatment/intervention group) and compare their outcomes to users who do not receive the same product or intervention (e.g., comparison group) to determine if there were differences between the groups. This is the most rigorous approach to understanding edtech product effectiveness, so we recommend building up to this type of RCE report by examining prior goals first. You can run this type of RCE report as often as you have outcomes available.

The outcome refers to the education outcome of interest, which should be a measure that the intervention could impact (e.g., examining literacy achievement when studying a literacy intervention). Typically, the outcome should be a quantitative measure with high reliability and validity, such as standardized testing scores. For outcomes analysis RCE reports, the analysis includes covariates that account for extraneous data points that might explain your outcome (e.g., pretest achievement).

Goal 5: Understand potential cost savings for my product

Are you interested in understanding potential cost savings for your product? Once you've analyzed the usage of your edtech product over time, you can identify opportunities for cost savings when products have low or limited use.

RCE reports can be performed to learn more about the overall population in your school or district or for various subpopulations (e.g., grade level, gender). The population is the group of participants (e.g., students) included in your analyses. The population of interest can be very broad or it can be specific to a particular subgroup of participants, which may include students within a given school or grade, students of differing performance levels, or students who identify as a given gender, ethnicity, or special needs status. For RCE reports, you may be able to create student groups based on your product implementation and the context for use in your organization. Your RCE Manager can help you determine what those groups may be.