![\"botchan-music\"](\"http:\/\/www.radicalfishgames.com\/wp-content\/uploads\/botchan-music2.png\")
It’s time for a new technical post!<\/p>\nIt’s time for a new technical post!<\/p>\n
As promised today, we want to share with you how we implemented music and sound in CrossCode<\/strong>. This includes how we used the WebAudio API<\/strong> to increase the sound quality and make perfect loops possible. Any code you will encounter is not directly from the game and might be pseudo-code to some extend.<\/p>\n So… grab some fish’n’chips and tea, this is going to be a long post!<\/p>\n For starters, the WebAudio API<\/strong> is a more or less high-level API for processing sound via JavaScript<\/strong>. Now that means that we can not only play sounds via the API, but we can also manipulate the sound in any way we want. To make this magic happen, the API uses a routing graph. You have a master node (called destination in the API) and attach a number of different nodes to it. Look at this figure showing how this visually looks like: AudioNode <\/strong>represents a single sound that will be played. As you can see there aren’t only AudioNodesthough. The WebAudio API<\/strong> features lots of nodes that make it possible to manipulate the actual sound. In the graph you can see a GainNode<\/strong>. This node is used to change the volume of the sound. Like any graph structure it is also possible to attach multiple nodes to another node. This can be really useful if you want to pass each sound through a single\u00a0GainNode <\/strong>to change the volume for all sounds. Developing Game Audio with the Web Audio API<\/a><\/p>\n This article features some very nice effects you can use and is a great start for learning how to use the API.<\/p>\n The API’s starting point is a single object that has all the functions to create different nodes:<\/p>\n For CrossCode <\/strong>we made a small list of things we wanted to have for both music and sound.<\/p>\n Lots of these points are already covered by the audio element<\/strong>. But the first 2 are hard to achieve. Perfect timing for music isn’t easily achieved and you might have noticed it too while playing CrossCode<\/strong>. The same goes for positioning sound in 3D. We might also come up with other ideas along the way that aren’t possible to do with the audio element<\/strong> so we started working on integrating the WebAudio API<\/strong>.<\/p>\n So, lets dive into the implementations! We’re going to begin with sound to explain some basic aspects of the API. We will also talk about the default implementation which is mostly covered by the music and sound classes impact.js<\/a> provides.<\/p>\n As said, impact.js<\/strong> covers a lot of ground here already. But as usual we changed a lot to make it work for us. One of these changes is to separate the default implementation, which uses the audio element<\/strong>, from the WebAudio<\/strong> Implementation. Normally we load a sound roughly like this:<\/p>\n And we wanted to keep it that way. So we have two sound implementations sharing the same interface, SoundWebKit and SoundDefault:<\/p>\n When the game loads, it will detect your browser and switch to the WebAudio API\u00a0<\/strong>if it is supported. For the default implementation we simply load an audio element<\/strong>. Since each audio element can’t play its sound multiple times in parallel, we need to load several audio elements of the same sound for parallel playback.\u00a0This is all handled nicely by impact.js<\/strong> so we don’t have to worry about it.<\/p>\n However for WebAudio <\/strong>we had to come up with something on our own. To load a sound via the API you mainly use a XMLHttpRequest<\/a>. But simply loading isn’t enough. We also need to decode the data we get from the request. This is done via the context mentioned in the WebAudio API<\/strong> earlier. Let’s simply assume that we have a global context created and ready to use. The code to load a sound would look like this:<\/p>\n As you can see there isn’t much effort in loading the sound file and decoding it. After we decoded the sound we cache the buffer in some global state, so we don’t need to load it again. This is also true for quickly repeating the same sound! The buffer will then be referenced in the WebAudio <\/strong>implementation of the sound object.<\/p>\n You’ll need to create a node each time you play the sound. That’s right, In contrast to the audio element<\/strong> we can’t simply reset the playback position to zero and play again:<\/p>\n As you can see we assign the buffer of the sound to the node. The AudioNode <\/strong>here is a so called AudioBufferSourceNode<\/strong>. As the name implies it is the base node for playing back any sound. We then need to connect the node to the destination node of the context. As explained at the beginning of this post, this way the sound gets routed through the audio graph. To play it we use the start\/noteOn<\/strong><\/em> method. Don’t get confused by the parameter. It is mandatory and is used as the offset time to start the playback.<\/p>\n But of course this is not all we do. We also want to position sound, right? For this we simply use another node that WebAudio <\/strong>already provides to set the position of a sound in 3D: the PannerNode<\/strong>. We now need to extend the method above to include a possible position:<\/p>\n For this method, we simply assume that every sound has a position. As you can see we create a new PannerNode <\/strong>here and connect it to the AudioBufferSourceNode<\/strong>. We then connect the position node to the destination and start playing the sound like before. In CrossCode <\/strong>we have a little bit extra code here which adjusts the position to be in world space. There is also a bit of code that makes sure that when you’re in a certain radius of an object that sound plays normally to create a smoother experience. Otherwise it would be very distracting if every sound makes a sudden gap from the right to the left speaker. This is basically what we do with WebAudio <\/strong>here. Of course not all browsers currently support WebAudio<\/strong>. So you might wonder what we do with positioned sound for those players. Of course we can’t achieve the same effect with the audio element<\/strong>, so we simulate the same effect via the volume of the sound. The further you away from the sound the softer it is. We hope that eventually all browsers will support the API so everyone can have positioned sound in their favorite browser.<\/p>\n And this is it for sound! Let’s summarize everything in a small list:<\/p>\nThe WebAudio API<\/h2>\n
\n![\"graph_simple\"](\"http:\/\/www.radicalfishgames.com\/wp-content\/uploads\/graph_simple.png\")
<\/a><\/p>\n
\nOther nodes make it possible to apply filters on a sound. Or you can use multiple sounds to cross-fade them based on your position in the game world. We don’t want to cover all the possibilities here since there already is a great article over at Html5Rocks<\/strong>. Check out this link:<\/p>\nwindow[\"AudioContext\"] = window[\"AudioContext\"] || window[\"webkitAudioContext\"];\r\nthis.context = new window[\"AudioContext\"]();<\/pre>\n
Goals for CrossCode<\/h2>\n
\n
Sound<\/h2>\n
var sound = new Sound(\"path\");<\/pre>\n
if(window[\"webkitAudioContext\"]) {\r\n Sound = SoundWebAudio;\r\n} else {\r\n Sound = SoundDefault;\r\n}<\/pre>\nvar request = new XMLHttpRequest();\r\nrequest.open('GET', path, true);\r\nrequest.responseType = 'arraybuffer';\r\nrequest.onload = function () {\r\n sound.context.decodeAudioData(request.response, function (buffer) {\r\n sound.buffers[path] = buffer;\r\n }, someFancyErrorHandler())\r\n}\r\nrequest.send();<\/pre>\nplay: function() {\r\n \/\/ creates a AudioNode called AudioBufferSourceNode\r\n var node = sound.context.createBufferSource(); \r\n node.buffer = this.buffer;\r\n\r\n node.connect(sound.context.destination);\r\n node.start(0); \/\/ or node.noteOn(0);\r\n}<\/pre>\nplay: function(pos) {\r\n var node = sound.context.createBufferSource();\r\n node.buffer = this.buffer;\r\n\r\n var position = sound.context.createPanner();\r\n position.setPosition(pos.x, pos.y);\r\n\r\n node.connect(position);\r\n position.connect(sound.context.destination);\r\n node.start(0); \/\/ or node.noteOn(0);\r\n\r\n return {pos: position, source: node};\r\n}<\/pre>\n
\nNotice that we also return an object here? In this pseudo-code it returns the position node and the source node. We do this to be able to adjust the position in real-time and be able to stop playback manually.<\/p>\n\n
Music<\/h2>\n