Scratch, Minecraft and Raspberry Pi: how they fit together

My son has been learning programming with Scratch. But moving sprites around the screen will only keep his attention for so long, so we've been trying to figure out how to get Scratch programs to manipulate Minecraft. 

The short answer

The short answer is  "Yes, you can". The longer is answer is more involved and complicated so I thought I'd collect my notes for others that are trying to figure it out.

The longer answer

Can you get Scratch to manipulate Minecraft? 
Well, it depends on which version of Scratch you want to use and also which version of Minecraft you want to use. 

To understand all the combinations and consequences you need to understand a bit about how they interact.

Scratch doesn't interact with Minecraft directly. You need to use a Scratch extension to interact with an external program. That external program then provides the interaction between Scratch and Minecraft.

The complication is that the extension to Scratch and the API/MOD for Minecraft are dependent on the versions of the respective applications you want to use.

Lets start with the easiest alternative - do everything on a Raspberry Pi.

The good news is that it just works. You can go and grab a copy of all the necessary bits at scratch2mcpi

The installer has all the bits you need. Just fire it up, then copy and modify some of the example applications.

There is also a 2nd package you can get which provides a Scratch programming wrapper to Martin O'Hanlon's python library for Minecraft Turtle Graphics. Again, the whole lot can be downloaded as a single installer and you can see the example projects on the author's Scratch area.

To use scratch for minecraft turtle graphics you need to open one of the examples (or the template project), modify it as you want and save as a new project. (This limitation is due to the way Scratch on the RPi deals with external programs... As we shall see)

The not so good news is that these two packages are kind of limited and you need to use an older version of Scratch. To understand the limitations lets have a look at how it works.

The version of Scratch you get on the RPi is v1.4 but the current version of Scratch is now v2.0.

Unfortunately the latest standalone version of Scratch doesn't run on the RPi because its built on Adobe Air which doesn't run on the RPi architecture. So you're stuck with v1.4. 

The limitation is that the way Scratch handles interaction with external programs is pretty limited in v1.4 but has received a nice upgrade in v2.0. 

Scratch 1.4 uses its Remote Sensors Protocol to communicate with external applications. This provides low-level socket communication between Broadcast messages from Scratch 1.4 and the external app. There is a small python project, ScratchPy which talks that protocol. The scratch2mcpi project builds on scratchpy to speak to Minecraft.

However, Minecraft on the RPi is not the same as regular Minecraft. Minecraft Pi Edition is a cut-down version similar to the version of minecraft that you get on your phone or tablet (or ps3/xbox console). It provides an API (both python and java) which enables external programs to interact with the world using the Minecraft Pi Protocol. This protocol isn't available on regular minecraft, just the RPi version (* there is an exception which I'll come back to...).

So, the available software only lets you interact between Minecraft and Scratch programs on the RPi. 

But what if you want to use Scratch v2.0 or a version of Minecraft that is not the Pi Edition? At the moment you can't. The only way to use Scratch 2.0 is to run it somewhere other than a RPi (Mac, Win, or Linux(x86)).

Scratch2 has a new mechanism for interacting with external programs - the Scratch Extension Protocol. At this time you need to use a standalone (not web) version of Scratch though there are plans to make it possible to use the protocol from the webapp version of the scratch editor.

The extension protocol speaks HTTP to a helper app running on the same machine as Scratch. That helper app can then interact with any other program. This approach is more flexible than the Scratch v1.4 mechanism that works for scratch2mcpi.

There isn't (or I can't find) a Scratch 2 extension app that interacts with Minecraft. But here is how it should work:

Lets start with the helper app. When you define a helper app you have the opportunity to define your own blocks that can be used in Scratch scripts. When these blocks are used in the scripts Scratch sends http messages to the helper app - simple GET messages with the operation and data from the block passed along. The helper app can then pass these along to Minecraft. Its also possible to have blocks that listen to the helper app for incoming information. Scratch continuously polls the helper app for new information so you could read info from Minecraft and use it in a script (not super efficient but it works...)

This is all much nicer than the simple Broadcast messages that are available in Scratch v1.4.

As an example, here is a screenshot of a test helper app I've been toying with to test out communication with Minecraft. The "post to chat" block will send a message via the helper app to the Minecraft Pi edition API to print a message to the players chat window. There is also a block which polls the app to return the players position to use in a script.

Now we can cover the options this creates on the Minecraft side.
The helper app is just a small http server which listens for HTTP requests from Scratch and you can then do what you like. There are a number of alternatives for interacting with Minecraft.

1. You could use the same mcpi api to communicate with Minecraft Pi Edition running on an RPi
2. Or you could connect to a full Minecraft server that has a MOD to provide an API to external clients. For example, you could write a http server in Node and then use the node-minecraft npm or the ScriptCraft mod to integrate to Minecraft. 
3. Finally you could actually use Minecraft Pi Protocol on a Minecraft server that isn't running on the RPi. The RaspberryJuice mod provides a slightly cut-down version of the Pi Protocol which allows you to integrate with the same API on both full servers and Pi Edition.

Next Steps

If you want to start using Scratch and Minecraft together, then you need to use one of the alternatives available on the Raspberry Pi. The next step is to see if I can get a Scratch2 helper app to expand those possibilities...