{"id":360,"date":"2020-10-06T10:24:24","date_gmt":"2020-10-06T10:24:24","guid":{"rendered":"https:\/\/machine-learning.webcloning.com\/2020\/10\/06\/collaborating-with-ai-to-create-bach-like-compositions-in-aws-deepcomposer\/"},"modified":"2020-10-06T10:24:24","modified_gmt":"2020-10-06T10:24:24","slug":"collaborating-with-ai-to-create-bach-like-compositions-in-aws-deepcomposer","status":"publish","type":"post","link":"https:\/\/salarydistribution.com\/machine-learning\/2020\/10\/06\/collaborating-with-ai-to-create-bach-like-compositions-in-aws-deepcomposer\/","title":{"rendered":"Collaborating with AI to create Bach-like compositions in AWS DeepComposer"},"content":{"rendered":"<div id=\"\">\n<p><a href=\"https:\/\/aws.amazon.com\/deepcomposer\/\" target=\"_blank\" rel=\"noopener noreferrer\">AWS DeepComposer<\/a> provides a creative and hands-on experience for learning generative AI and machine learning (ML). We recently launched the <strong>Edit melody<\/strong> feature, which allows you to add, remove, or edit specific notes, giving you full control of the pitch, length, and timing for each note. In this post, you can learn to use the <strong>Edit melody<\/strong> feature to collaborate with the <a href=\"https:\/\/aws.amazon.com\/about-aws\/whats-new\/2020\/06\/aws-deepcomposer-announces-autoregressive-cnn-algorithm-for-generating-compositions-and-an-associated-learning-capsule-to-understand-the-machine-learning-concepts\/\" target=\"_blank\" rel=\"noopener noreferrer\">autoregressive convolutional neural network (AR-CNN)<\/a> algorithm and create interesting Bach-style compositions.<\/p>\n<p>Through human-AI collaboration, we can surpass what humans and AI systems can create independently. For example, you can seek inspiration from AI to create art or music outside their area of expertise or offload the more routine tasks, like creating variations on a melody, and focus on the more interesting and creative tasks. Alternatively, you can assist the AI by correcting mistakes or removing artifacts it creates. You can also influence the output generated by the AI system by controlling the various training and inference parameters.<\/p>\n<p>You can co-create music in the AWS DeepComposer Music Studio by collaborating with the AI (AR-CNN) model using the <strong>Edit melody<\/strong> feature. The AR-CNN Bach model modifies a melody note by note to guide the track towards sounding more Bach-like. You can modify four advanced parameters when you perform inference to influence how the input melody is modified:<\/p>\n<ul>\n<li>\n<strong>Maximum notes to add <\/strong>\u2013 Changes the maximum number of notes added to your original melody<\/li>\n<li>\n<strong>Maximum notes to remove <\/strong>\u2013 Changes the maximum number of notes removed from your original melody<\/li>\n<li>\n<strong>Sampling iterations <\/strong>\u2013 Changes the exact number of times you add or remove a note based on note-likelihood distributions inferred by the model<\/li>\n<li>\n<strong>Creative risk <\/strong>\u2013 Allows the AI model to deviate from creating Bach-like harmonies<\/li>\n<\/ul>\n<p>The values you choose directly impact the composition created by the model by nudging the model in one way or another. For more information about these parameters, see <a href=\"https:\/\/console.aws.amazon.com\/deepcomposer\/home?region=us-east-1#learningCapsules\/autoregressive\" target=\"_blank\" rel=\"noopener noreferrer\">AWS DeepComposer Learning Capsule on using the AR-CNN model<\/a>.<\/p>\n<p>Although the advanced parameters allow you to guide the output the AR-CNN model creates, they don\u2019t provide note-level control over the music produced. For example, the AR-CNN model allows you to control the number of notes to add or remove during inference, but you don\u2019t have control over the exact notes the model adds or removes.<\/p>\n<p>The <strong>Edit melody<\/strong> feature bridges this gap by providing an interactive view of the generated melody so you can add missing notes, remove out-of-tune notes, or even change a note\u2019s pitch and length. This granular level of editing facilitates better human-AI collaboration. It enables you to correct mistakes the model makes and harmonize the output to your liking, giving you more ownership of the creation process.<\/p>\n<p>For this post, we explore the use case of co-creating Bach-like background music to match the following video.<\/p>\n<h2>Collaborating with AI using the AWS DeepComposer Music Studio<\/h2>\n<p>To start composing your melody, complete the following steps:<\/p>\n<ol>\n<li>Open the <a href=\"https:\/\/console.aws.amazon.com\/deepcomposer\/home?region=us-east-1#musicStudio\" target=\"_blank\" rel=\"noopener noreferrer\">AWS DeepComposer Music Studio<\/a> console.<\/li>\n<li>Choose an <strong>Input melody<\/strong>.<\/li>\n<\/ol>\n<p>You can record a custom melody, import a melody, or choose a sample melody on the console. \u00a0For this post, we experimented with two melodies: the <em>New World<\/em> sample melody and a custom melody we created using the MIDI keyboard.<\/p>\n<p><em>New World<\/em> melody:<\/p>\n<p>       <audio class=\"wp-audio-shortcode\" id=\"audio-16716-1\" preload=\"none\" controls=\"controls\"><\/audio><\/p>\n<p>Custom melody:<\/p>\n<p>       <audio class=\"wp-audio-shortcode\" id=\"audio-16716-2\" preload=\"none\" controls=\"controls\"><\/audio><\/p>\n<ol start=\"3\">\n<li>Choose the <strong>Autoregressive<\/strong> generative AI technique.<\/li>\n<li>Choose the <strong>Autoregressive CNN Bach <\/strong>model.<\/li>\n<\/ol>\n<p>There are several considerations when choosing the advanced parameters. First, we wanted the original input melody to be recognizable. After some iterating, we found that setting the <strong>Maximum notes to add <\/strong>to 60 and <strong>Maximum notes to remove <\/strong>to 40 created a desirable outcome. For <strong>Creative risk<\/strong>, we wanted the model to create something interesting and adventurous. At the same time, we realized that a very high <strong>Creative risk <\/strong>value would deviate too much from the Bach style, so we took a moderate approach and chose a <strong>Creative risk <\/strong>of 2.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-16720\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/10\/02\/1-.jpg\" alt=\"\" width=\"900\" height=\"413\"><\/p>\n<ol start=\"5\">\n<li>You can repeat these steps a few times to iteratively create music.<\/li>\n<\/ol>\n<h2>Editing your input melody<\/h2>\n<p>After the AR-CNN model has generated a composition to your satisfaction, you can use the <strong>Edit melody<\/strong> feature to modify the melody and try to match the video\u2019s transitions as much as possible.<\/p>\n<ol>\n<li>Choose the right arrow to open the input melody section.<\/li>\n<li>Choose <strong>Edit melody<\/strong>.<\/li>\n<li>On the <strong>Edit melody<\/strong> page, edit your track in any of the following ways:\n<ul>\n<li>Choose a cell (double-click) to add or remove a note at that pitch or time.<\/li>\n<li>Drag a cell up or down to change a note\u2019s pitch.<\/li>\n<li>Drag the edge of a cell left or right to change a note\u2019s length.<\/li>\n<\/ul>\n<\/li>\n<li>When finished, choose <strong>Apply changes<\/strong>.<\/li>\n<\/ol>\n<p>We drew inspiration from the AI-generated notes in different ways. For the <em>New World<\/em> melody, we noticed the model added short and bouncy notes (the circles with solid lines in the following screenshot), which made the composition sound similar to an American folk song. To match that style, we added a few notes in the second half of the composition (the dotted-lined circles).<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-16721\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/10\/02\/2-.jpg\" alt=\"\" width=\"900\" height=\"549\"><\/p>\n<p>For our custom melody, we noticed the model changed the chords slightly earlier than expected (see the following screenshot). This created lingering and overlapping sounds that we liked for the mountain road scenes.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-16722\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/10\/02\/3-.jpg\" alt=\"\" width=\"900\" height=\"546\"><\/p>\n<p>On the other hand, we noticed the AI model needed our help to remove some notes that sounded out of place. After we listened to the track a few times, we decided to change some pitches manually to nudge the track towards something that sounded a bit more harmonious.<\/p>\n<h2>Generating accompaniments using the GAN generative AI technique<\/h2>\n<p>After using the AR-CNN Bach model to explore options for our melody track, we decided to try using a different generative AI model (GAN) to create musical accompaniments.<\/p>\n<ol>\n<li>Under <strong>Model parameters<\/strong>, for<strong> Generative AI technique<\/strong>, choose <strong>Generative adversarial network<\/strong>.<\/li>\n<li>Feed the edited compositions to the GAN model to generate accompaniments.<\/li>\n<\/ol>\n<p>We chose the <strong>MuseGAN<\/strong> generative algorithm and the <strong>Symphony<\/strong> model because we wanted to create accompaniments to match the serene and somber setting in the video.<\/p>\n<p><strong><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-16723\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/10\/02\/4-.jpg\" alt=\"\" width=\"900\" height=\"466\"><\/strong><\/p>\n<ol start=\"3\">\n<li>You can optionally export your compositions into a music-editing tool of your choice to change the instrument set and perform post-processing.<\/li>\n<\/ol>\n<p>Let\u2019s watch the videos containing our AI-inspired creations in the background.<\/p>\n<p>The first video uses the <em>New World <\/em>melody.<\/p>\n<p>The following video uses our custom melody.<\/p>\n<h2>Conclusion<\/h2>\n<p>In this post, we demonstrated how to use the <strong>Edit melody<\/strong> feature in the AWS DeepComposer Music Studio to collaborate with generative AI models and create interesting Bach-style compositions. You can modify a melody to your liking by adding, removing, and editing specific notes. This gives you full control of the pitch, length, and timing for each note to produce an original melody.<\/p>\n<hr>\n<h3>About the Authors<\/h3>\n<p><strong>\u00a0<img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-16725 alignleft\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/10\/02\/RahulSuresh.jpg\" alt=\"\" width=\"100\" height=\"138\">Rahul Suresh<\/strong> is an Engineering Manager with the AWS AI org, where he has been working on AI based products for making machine learning accessible for all developers. Prior to joining AWS, Rahul was a Senior Software Developer at Amazon Devices and helped launch highly successful smart home products. Rahul is passionate about building machine learning systems at scale and is always looking for getting these advanced technologies in the hands of customers. In addition to his professional career, Rahul is an avid reader and a history buff.<\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><strong><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-16724 alignleft\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/10\/02\/enocchen.jpg\" alt=\"\" width=\"100\" height=\"134\"><\/strong><strong>Enoch Chen<\/strong> is a Senior Technical Program Manager for AWS AI Devices. He is a big fan of machine learning and loves to explore innovative AI applications. Recently he helped bring DeepComposer to thousands of developers. Outside of work, Enoch enjoys playing piano and listening to classical music.<\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-16764 alignleft\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/10\/05\/carlosdaccarett.jpg\" alt=\"\" width=\"98\" height=\"133\"><strong>Carlos Daccarett<\/strong> is a Front-End Engineer at AWS. He loves bringing design mocks to life. In his spare time, he enjoys hiking, golfing, and snowboarding.<\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p>\u00a0<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-16765 alignleft\" src=\"https:\/\/d2908q01vomqb2.cloudfront.net\/f1f836cb4ea6efb2a0b1b99f41ad8b103eff4b59\/2020\/10\/05\/jacdylan_cropped.jpg\" alt=\"\" width=\"100\" height=\"140\"><strong>Dylan Jackson<\/strong> is a Senior ML Engineer and AI Researcher at AWS. He works to build experiences which facilitate the exploration of AI\/ML, making new and exciting techniques accessible to all developers. Before AWS, Dylan was a Senior Software Developer at Goodreads where he leveraged both a full-stack engineering and machine learning skillset to protect millions of readers from spam, high-volume robotic traffic, and scaling bottlenecks. Dylan is passionate about exploring both the theoretical underpinnings and the real-world impact of AI\/ML systems. In addition to his professional career, he enjoys reading, cooking, and working on small crafts projects.<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/aws.amazon.com\/blogs\/machine-learning\/collaborating-with-ai-to-create-bach-like-compositions-in-aws-deepcomposer\/<\/p>\n","protected":false},"author":0,"featured_media":361,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/360"}],"collection":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/comments?post=360"}],"version-history":[{"count":0,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/posts\/360\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media\/361"}],"wp:attachment":[{"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/media?parent=360"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/categories?post=360"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/salarydistribution.com\/machine-learning\/wp-json\/wp\/v2\/tags?post=360"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}