Here's What 44 million Scratch Scripts Look Like

Monday, August 08, 2011

In preparation for our 2 million project mark celebration and as part of my research on remixing, I have been analyzing the use and reuse of components in the Scratch Online Community. For example, I have looked into which images and programming blocks are more commonly used. Now I wanted to go one step further. I wanted to know what are the most common programming constructs or scripts created by the young Scratch programmers. So here it is, a word cloud-like representation of the 100 most common scripts.

Click for larger version

Looks Matter
By far, the most common scripts involve some kind of looks manipulation such as hiding/showing a sprite and switching its costumes. This is probably because controlling what is displayed on the screen is useful and necessary for most types of projects, from games to animations. Also, these scripts often come in pairs: for every "hide" I would expect a "show".

1st place, 9.16%

The most commonly used script (9.16% of the total) is a two block script that hides a sprite when an event occurs. The names of the events vary widely. But just to give you a an idea of the types of events we're talking about, the most common events that trigger this script are "Game Over" (2.54%) and "start" (2.44%).

Below you will see a list of the most common scripts that have something to do with looks, as well as their position in the ranking and percentage of the total, both based on their frequency.

2nd place, 4.84%

 

4th place, 3.28%

6th place, 1.36%

8th place, 0.95%

9th place, 0.73%

11th place, 0.63%

14th place, 0.53%

15th place, 0.47%

17th place, 0.42%

18th place, 0.40%

21st, 0.35%

As you can see from the small percentages, the frequency distribution of scripts appears to be a long tail distribution. This is to be expected given the large number of combinations that are possible. One might expect a similar distribution if we were to look for the most popular phrases in the English language (probably an even longer and flatter tail).

Interacting via the Keyboard

16th place, 0.46%

It is nice to see that interactivity ranked highly as well. After all, interactivity is one of the features that distinguishes Scratch projects from, say, videos or pictures. You can see that some of the scripts above involve interactivity. For example, the 11th most common script is probably used in interactive stories that function like slideshows. I say this partly because I have seen this quite often and because the most commonly used keys that trigger this script are "space" (36.55%) and "1" (6.66%).  The use of the "space" key to interact with projects has developed into a cultural norm that participants learn in the Scratch Online Community (possibly influenced by Microsoft PowerPoint as well).

While slideshows are interactive, we can see even more complex interactivity in the 16th most popular script. This script is often used in games to let a player control a character using the keyboard. The most common arguments used are "right arrow ➡,  direction 90° and a move 10 steps" (16.21%) followed by the equivalent "left arrow  ⬅ and direction -90°" (16.81%).

27th  0.25%
(with sample arguments)

Update - I was asked about script #27. Here is what I found.
Despite not being obviously interactive, the 27th most common script represents a form of interactivity because one of its arguments is a variable changed by pressing the arrow keys. As we can see in this this project (the very first one to use this script), these blocks are typically used to control the horizontal position of background elements on on a scrolling background game.

Background Sound

13th place

I was a bit surprised to find a script related to sound ranked so highly. I guess both animations and games often have some sound playing continuously in the background. Looking more closely, I was even more surprised to find that the sounds looped more frequently are not music files imported into Scratch (i.e. commercial songs) but recordings created within Scratch using the microphone. The most common sound name played with this script is "recording1" (3.82%) followed by "one1" (1.08%).

Signs of Experimentation
You will notice that some of the scripts in the script cloud are single hat blocks. I was debating whether to include them or not. Technically, I considered them to be scripts even if they don't have any other blocks underneath. I decided to include them because it is quite telling how often people drag a hat block and leave it unused. Compared to other languages, Scratch is quite forgiving and lets people do this without any big repercussions. I would like to think these unused hat blocks represent moments of tinkering and experimentation, something that we value a lot in Scratch.

3rd, 4.73%

5th, 1.44%

12th, 0.63%

10th place, 0.68%

After talking to some people, I did decide to leave out the script that had the comment block by itself. For several technical reasons it was identified as a script by the analysis I ran and it's in the 10th position in the list (0.68%). The use of the comment block was both surprising and encouraging. Partly because it was added recently so a lot of projects back in 2007 and 2008 did not even have the option of using it.

Methodology

Equivalent scripts.
Arguments are ignored

For the past few years, I have been collecting a massive database with information about the components of each version of every project uploaded to the Scratch website. Among other things, this database has the human-readable representation of the scripts for each sprite. As you might know, a sprite can have zero or more scripts, so I started by extracting each script associated with every sprite and created a new database table for it. This new table has a record for every script that comes with its the human-readable version, the id of the project, its version number and the id of the sprite, among other fields.  As I did this, I also added a column to store a version of the script without any arguments. This way, scripts with different arguments but with the same block sequence are considered as equals. This new database table of scripts has more than 44 million scripts and close to 1.7 million unique projects out of a total of 1,883,872 projects (90% of projects have scripts). If you are interested in looking at the data, please check out this spreadsheet and let us know if you find any other facts worth mention (or any mistakes!).

This analysis was possible thanks to the work of former MIT student Rita Chen and members of the Scratch community including MyRedNeptune and the active Scratch Wiki editors Jonathanpb, Scimonster and BWOG. Thanks gals and guys!

Scratch 2.0: What are you searching for?

Sunday, July 31, 2011

Finding the project you’re looking for in a collection of 100 projects is pretty easy. But finding one project in 2,000,000 can be hard. When you add in all the other things you might want to search for -- forum posts, galleries, wiki entries, and support pages -- then search can get pretty tricky. So as we’ve worked on Scratch 2.0, we decided to invest some energy on improving the search tools in Scratch.

We started with some rough conceptual mockups. Like most designers, we make a lot of “mockups” - models that show different ways that something might work. Mockups are easy and quick to make, because they don’t have to look good, and they don’t even have to function. All they have to do is show how an approach or an idea might work so that the design team can think it over, weigh the advantages and the disadvantages, and come up with new ideas.

This is the first mockup we made when we started to think about ways to improve search. It shows what the results for a search on “lolcats” might look like. Forum posts are at the top, followed by projects, galleries, wiki pages, and tags. This is a different way of organizing the search results than the approach used by the current Scratch website, which just displays a list and lets you filter the results by type afterwards.

Gaia joins the Scratch Team

Shortly after we made this initial mockup, Gaia Carini joined the Scratch Team and dove into the problem of search. Gaia recently graduated from MIT, and chose to come back to earn a Masters of Engineering degree. She’s made many mockups and several working prototypes so that the design team could try out different approaches. Here’s a picture of her latest (but not final) search interface.

This shows a few of the features that have grown out of discussions about past prototypes, like the ability to enlarge and play a project right on the search page (without having to click-through to the project page). Links on the left sidebar help you refine your initial results, so you can narrow things down with only a few clicks. This makes finding what you’re looking for a lot quicker and easier.

We're still considering additional ways to enhance search. For example, should we include an "advanced search" interface, to allow for deeper searches? Should the search box automatically suggest search terms as you type them in? And we're still not quite sure if it's better to display results in separate categories, as in the first mockup, or if we should display them as a mix of projects, forum posts, etc.

What do you search for on the Scratch website? Are there things you wish you could find more easily? Do you have suggestions on how to improve search in Scratch? Share your thoughts in this forum thread.

Scratch 2.0: Friend or Follow?

Friday, June 24, 2011

When we developed the Scratch website, we decided to make friending another Scratcher non-symmetrical. That’s just a fancy way of saying you can add someone to your list of friends without them having to friend you.

For example, let’s say you want to friend a Scratcher called Gobo. You can do this by visiting Gobo’s My Stuff page and clicking Add to friends. Gobo will then show up on your list of friends. However, that won’t make your username appear on Gobo’s friend’s list. Gobo can always go to your My Stuff page and add you as a friend too. But until they do, Gobo is your friend, but you aren’t Gobo’s friend.

On Facebook and most other social networking sites, friending works a little differently. When you invite someone to be your friend, the site sends them a message asking if they’d like to be friends with you. If they don’t approve the request (for instance, if they don’t know you well enough to be friends yet), nothing happens. If they do approve the request, then they will be on your friend’s list and you will also be on theirs.

As we work on Scratch 2.0, we have the opportunity to re-visit the choices we made with the first version of the website, and decide to keep things the same or do things differently. For example, we could make friending on Scratch work more like it does on Facebook. This might avoid some confusion, since it seems like some people who are new to Scratch assume that friending on Scratch works like friending on Facebook.

If we made that change, then we might also want to make it possible for Scratchers to “follow” or “watch” other Scratchers that they think make interesting projects. Following would work the way friending works on Scratch now. You can follow anyone you like, but following someone doesn’t make them follow you. When someone you are following makes a new project, you get an update in your messages with a link. There could also be a row on the front page that shows recent projects by people you are following.

In this scenario, you could follow Scratchers that you think make interesting stuff -- but you would friend Scratchers that you know a little better, perhaps because you’ve gotten to know each other over time.

What do you think? Is it worth changing the friend system on Scratch? Or should we just keep the friends system the same? Join in the discussion on this forum thread and share your thoughts.

p.s. You can check out a related discussion and suggestion started by cheddargirl on this forum thread.

Scratch 2.0: Moving to the Cloud

Tuesday, May 10, 2011

Scratch 2.0 will be “all in the cloud” -- meaning that the Scratch programming editor will become part of the Scratch website. We’re excited about this transition because it has a lot of great benefits. For one thing, it won’t be necessary to download and install Scratch to try it out. For another, it will be easier to see the scripts that show how a project works. Instead of downloading the project file, all you need to do is click a button to see inside and play with the code -- then click another button to remix it. Once the transition to “the cloud” is complete, it’ll be easier for us to continually add new features to Scratch, without releasing an entire new version.

But moving the Scratch editor to the website raises some tough questions. For example:

Should you be able to see other people’s Scratch projects that aren’t yet finished?

If a Scratcher starts a new project and gets halfway through before stopping to do something else, should others be able to see their work in progress? Or should it be hidden until the Scratcher decides they are ready to share it with everyone else?

There are advantages and disadvantages to both approaches. Being able to see a project that’s only halfway finished could ruin the surprise of a newly released project. On the other hand, some Scratchers might like to get constructive advice and feedback on their unfinished, draft projects.

Right now, we’re imagining that Scratchers will work on their projects in their own private section of the Scratch website. Their project will only become visible when they click a button to “share” it with the rest of the Scratch community. But because we’re still in the early stages of developing Scratch 2.0, this could still change.

What do you think? Should Scratchers be able to see each other’s draft projects? Post your thoughts on this forum thread.

Edit: Don't worry, we are still planning to make a downloadable Scratch application, so it will still be possible to work on projects "offline." See more about this in the Scratch 2.0 FAQ.

Scratch 2.0: Create your own block

Saturday, March 26, 2011

Flash Player improvements

Thanks to everyone who reported bugs in the new flash-based Scratch project player! We’re now up to version 23 (the third version since the original release), and so far we’ve fixed over 20 confirmed glitches. Lots of Scratchers have posted detailed bug reports on the forums, which is really helpful. The more glitches we can find and fix now, the better we can make the flash player (and the next version of Scratch).

Create your own block

One of the new features we’re excited to add to Scratch is the ability to create your own blocks -- called “procedures” in the language of computer science. Creating your own blocks can be really handy. For example, let’s say you want to create an animation with a sprite that jumps in the air. Each time the sprite jumps, you need to tell it to go up, wait a little while, and then come back down again. That takes at least 3 blocks for every jump.

A simple “jump” script.

If you want to make your sprite jump, move, and then jump again, you might make something like this.

A “jump, move, jump” script.

In Scratch 2.0, you can do the same thing by creating your own jump block. You tell Scratch what your jump block does by creating a special script that defines the new block:

Creating a jump block to make a “jump, move, jump” script.

In this example, the jump block makes the sprite jump up, wait a half second, and then come down again. Each time Scratch comes to a jump block in a script (like the script on the left), Scratch runs the commands in the jump definition script (on the right). Combining the commands for jumping into a single block makes scripts easier to read.

In the next version of your project, you might want to be able to control how high your sprite jumps. Without the ability to make your own jump block, that could be a real pain. You’d have to find each and every “change y by 50” block that was used for jumping in your project, and change the 50 to something else. But with the new create-your-own-block feature, you can just add an input that tells the jump block how high you want it to jump.

Creating a jump block with a “height” input.

Now each time the jump block is run, the value of the height variable will be set to whatever number is entered as an input on the jump block. In this example, the first jump block has an input of 25, so the height variable will be set to 25 and the sprite will jump 25 pixels high. The second jump block has an input of 50, so the sprite will jump 50 pixels high.

We’re still in the process of figuring out the best way for Scratchers to create their own blocks, and there are still many questions. For example, if you create a jump block in one sprite, would other sprites be able to use the jump block too? Should the definition script appear in only one sprite -- and, if so, how would people find it? Should you be able to define “jump” differently in different sprites? These are few of the questions we have to think through before we’re ready to put this cool feature in the next version of Scratch.

Have ideas or thoughts about this? Post them in this thread in the Scratch forums.

Top 10 images in Scratch

Monday, February 14, 2011

There are more than 1.5 million projects today on the Scratch website containing more than 45 million images and sounds. Here is the list of the 10 most common images used. The number in parenthesis represents the number of times that image is used.

1. button (58,072)
2. cat1-a (35,748)
3. bananas1 (33,880)
4. underwater (26,646)
   
5. beachball1 (25,694)
6. spotlight-stage (22,924)
   
7. bat1-a (21,220)
   
8. buttonPressed (20,973)
   
9. gobo1 (20,176)
10. shark1-b (19,368)

Methodology

Each Scratch project can have one or more versions uploaded to the website (only the latest one is visible to the public). Each project version has some sprites and a stage. Each sprite and each stage can have one or more images.  Whenever a project gets uploaded, it gets analyzed and the attributes of its different components (blocks, images, sounds, etc) get stored in a structured database. In that database we have a table with the metadata about each image and sound, such as its name and size. I used that table to find out what are the most common images based on their name.

In order to keep things simple and since most projects have only one version, I decided to ignore version 2 and higher. Then I grouped the images by name, assuming that two images with the same name (e.g. shark1-b) were the same. Of course, this assumption is not always correct. For example, it means that if someone imported an image called awesome-cat.png (Scratch would give it the name "awesome-cat" once imported) and then edited it, it would get counted along with an unedited version of the same image. Another challenge is that whenever people use the Scratch paint editor, images get assigned default sequential names (e.g. costume8, costume9, etc). So names like costume1and background1, along with their equivalents in other languages (e.g. disfraz1 in Spanish), are unlikley to represent the same image. For this reason, I ignored all those non-unique image names from the list of the top 1000 most common image names . That included names like "normal" or "1" which after a manual analysis of a small sample proved to also represent a wide variety of images. In the future, a more accurate analysis would involve parsing all the projects and generate a hash of the binary representation of each image.

It is not surprising that all of the images in this list are images that come with Scratch itself. However, they represent less than one percent (1%). The reality is that the the distribution of images follows a distribution with a long tail where there are a lot of images that get used once or twice.

Scratch 2.0 Progress Report

Friday, February 11, 2011

This is the first in a series of monthly Scratch 2.0 updates. Each update will highlight a few new features that we’re considering for Scratch 2.0.

Flash-Based Scratch Player

Ever notice how some projects act differently on the website than they do in the Scratch 1.4 application? That’s because the software that plays your Scratch project in the Scratch 1.4 application is different from the software that plays the project on the website. That’s one reason why we decided to make the transition to a Flash-based Scratch player and editor in Scratch 2.0. Both the player and the editor (where you create your scripts) will run inside your web browser, and both will be based on Flash, so your projects will run the same in both places.

Soon we’ll be releasing a beta version of our Flash-based Scratch player to the website, so that you can test it out. This new player includes some cool new features, like the ability to make the project completely fill your web-browser’s window with one click. Once it is ready, the link to try out the new player will appear on each project page.

The new icon at the top-left of the player is for full-window mode

We want the Flash player to work well with all Scratch projects, so once the beta is released we’ll need your help testing and reporting bugs!

Customizable User Pages

We’ve been really excited by all the great suggestions, forum discussions, and mockups (here and here) of ideas for my-stuff pages in Scratch 2.0. Based on these suggestions, we’ve decided to allow Scratchers to customize their user pages around a set of “widgets,” similar to the way that Deviant Art works. For example, each Scratcher can have a small box or “widget” on their user page to feature a few of their own projects. Another widget could feature their projects with the most views. The image below shows an early mockup of the editor where you can arrange widgets for your user page. This will allow Scratchers to customize their user pages in lots of interesting ways, which is currently the second most popular Scratch Suggestion.

So far we’ve been focused entirely on the functionality of Scratch 2.0, and haven’t even started thinking about the look and feel, so the final product is likely to look very different from this mockup.

Keep it Simple!

As always, our goal is to keep things simple: We want to make sure it’s easy for someone with no experience programming to dive in and get started with Scratch. Of course, we also want them to be able to make all kinds of cool stuff as they learn more about Scratch. So we spend a lot of time thinking about how to make Scratch both simple and powerful.

In developing Scratch 2.0, we’re starting by making a version of the Scratch editor work in the browser, with the same features and capabilities as the current Scratch 1.4 application. Once that is complete, we’ll begin adding new features — but making sure to keep everything simple. Here are a few ideas we’re working on: the ability to create your own blocks (thanks to the BYOB folks for thinking this one through!), hide/show lists, better tools for collaboration, ways to pull data from the web, and many others ideas marked “under review” and “planned” on the Scratch Suggestions site.

Look for the next Scratch 2.0 update in March, when we’ll describe our plans for improved support for collaboration on Scratch.