Hacking our solution to higher group conferences

Summarization header image

As somebody who takes loads of notes, I’m all the time looking out for instruments and techniques that may assist me to refine my very own note-taking course of (such because the Cornell Methodology). And whereas I typically want pen and paper (as a result of it’s proven to assist with retention and synthesis), there’s no denying that know-how might help to reinforce our built-up talents. That is very true in conditions equivalent to conferences, the place actively collaborating and taking notes on the similar time could be in battle with each other. The distraction of trying right down to jot down notes or tapping away on the keyboard could make it arduous to remain engaged within the dialog, because it forces us to make fast selections about what particulars are vital, and there’s all the time the chance of lacking vital particulars whereas attempting to seize earlier ones. To not point out, when confronted with back-to-back-to-back conferences, the problem of summarizing and extracting vital particulars from pages of notes is compounding – and when thought-about at a bunch degree, there may be significant individual and group time waste in fashionable enterprise with most of these administrative overhead.

Confronted with these issues every day, my group – a small tiger group I wish to name OCTO (Workplace of the CTO) – noticed a possibility to make use of AI to enhance our group conferences. They’ve developed a easy, and simple proof of idea for ourselves, that makes use of AWS providers like Lambda, Transcribe, and Bedrock to transcribe and summarize our digital group conferences. It permits us to assemble notes from our conferences, however keep targeted on the dialog itself, because the granular particulars of the dialogue are routinely captured (it even creates a listing of to-dos). And at present, we’re open sourcing the software, which our group calls “Distill”, within the hopes that others may discover this convenient as effectively: https://github.com/aws-samples/amazon-bedrock-audio-summarizer.

On this put up, I’ll stroll you thru the high-level structure of our venture, the way it works, and provide you with a preview of how I’ve been working alongside Amazon Q Developer to show Distill right into a Rust CLI.

The anatomy of a easy audio summarization app

The app itself is simple — and this is intentional. I subscribe to the idea that systems should be made as simple as possible, but no simpler. First, we upload an audio file of our meeting to an S3 bucket. Then an S3 trigger notifies a Lambda function, which initiates the transcription process. An Event Bridge rule is used to automatically invoke a second Lambda function when any Transcribe job beginning with summarizer- has a newly updated status of COMPLETED. Once the transcription is complete, this Lambda function takes the transcript and sends it with an instruction prompt to Bedrock to create a summary. In our case, we’re using Claude 3 Sonnet for inference, but you can adapt the code to use any model available to you in Bedrock. When inference is complete, the summary of our meeting — including high-level takeaways and any to-dos — is stored back in our S3 bucket.

Distill architecture diagram

I’ve spoken many times about the importance of treating infrastructure as code, and as such, we’ve used the AWS CDK to manage this project’s infrastructure. The CDK gives us a reliable, consistent way to deploy resources, and ensure that infrastructure is sharable to anyone. Beyond that, it also gave us a good way to rapidly iterate on our ideas.

Using Distill

If you try this (and I hope that you will), the setup is quick. Clone the repo, and observe the steps within the README to deploy the app infrastructure to your account utilizing the CDK. After that, there are two methods to make use of the software:

  1. Drop an audio file straight into the supply folder of the S3 bucket created for you, wait a couple of minutes, then view the ends in the processed folder.
  2. Use the Jupyter pocket book we put collectively to step by way of the method of importing audio, monitoring the transcription, and retrieving the audio abstract.

Right here’s an instance output (minimally sanitized) from a latest OCTO group assembly that solely a part of the group was capable of attend:

Here’s a abstract of the dialog in readable paragraphs:

The group mentioned potential content material concepts and approaches for upcoming occasions like VivaTech, and re:Invent. There have been strategies round keynotes versus having hearth chats or panel discussions. The significance of crafting thought-provoking upcoming occasions was emphasised.

Recapping Werner’s latest Asia tour, the group mirrored on the highlights like partaking with native college college students, builders, startups, and underserved communities. Indonesia’s initiatives round incapacity inclusion have been praised. Helpful suggestions was shared on logistics, balancing work with downtime, and optimum occasion codecs for Werner. The group plans to research turning these learnings into an inner publication.

Different subjects coated included upcoming advisory conferences, which Jeff might attend just about, and the evolving position of the fashionable CTO with elevated deal with social affect and world views.

Key motion gadgets:

  • Reschedule group assembly to subsequent week
  • Lisa to flow into upcoming advisory assembly agenda when out there
  • Roger to draft potential panel questions for VivaTech
  • Discover recording/streaming choices for VivaTech panel
  • Decide content material possession between groups for summarizing Asia tour highlights

What’s extra, the group has created a Slack webhook that routinely posts these summaries to a group channel, in order that those that couldn’t attend can atone for what was mentioned and rapidly overview motion gadgets.

Bear in mind, AI is just not excellent. A few of the summaries we get again, the above included, have errors that want guide adjustment. However that’s okay, as a result of it nonetheless hurries up our processes. It’s merely a reminder that we should nonetheless be discerning and concerned within the course of. Crucial considering is as vital now because it has ever been.

There’s worth in chipping away at on a regular basis issues

This is just one example of a simple app that can be built quickly, deployed in the cloud, and lead to organizational efficiencies. Depending on which study you look at, around 30% of corporate employees say that they don’t complete their action items because they can’t remember key information from meetings. We can start to chip away at stats like that by having tailored notes delivered to you immediately after a meeting, or an assistant that automatically creates work items from a meeting and assigns them to the right person. It’s not always about solving the “big” problem in one swoop with technology. Sometimes it’s about chipping away at everyday problems. Finding simple solutions that become the foundation for incremental and meaningful innovation.

I’m particularly interested in where this goes next. We now live in a world where an AI powered bot can sit on your calls and can act in real time. Taking notes, answering questions, tracking tasks, removing PII, even looking things up that would have otherwise been distracting and slowing down the call while one individual tried to find the data. By sharing our simple app, the intention isn’t to show off “something shiny and new”, it’s to show you that if we can build it, so can you. And I’m curious to see how the open-source community will use it. How they’ll extend it. What they’ll create on top of it. And this is what I find really exciting — the potential for simple AI-based tools to help us in more and more ways. Not as replacements for human ingenuity, but aides that make us better.

To that end, working on this project with my team has inspired me to take on my own pet project: turning this tool into a Rust CLI.

Building a Rust CLI from scratch

I blame Marc Brooker and Colm MacCárthaigh for turning me right into a Rust fanatic. I’m a methods programmer at coronary heart, and that coronary heart began to beat lots quicker the extra acquainted I acquired with the language. And it grew to become much more vital to me after coming throughout Rui Pereira’s wonderful research on the vitality, time, and reminiscence consumption of various programming languages, once I realized it’s great potential to assist us construct extra sustainably within the cloud.

Throughout our experiments with Distill, we needed to see what impact transferring a perform from Python to Rust would appear to be. With the CDK, it was simple to make a fast change to our stack that allow us transfer a Lambda perform to the AL2023 runtime, then deploy a Rust-based model of the code. In the event you’re curious, the perform averaged chilly begins that have been 12x quicker (34ms vs 410ms) and used 73% much less reminiscence (21MB vs 79MB) than its Python variant. Impressed, I made a decision to essentially get my fingers soiled. I used to be going to show this venture right into a command line utility, and put a few of what I’ve realized in Ken Youens-Clark’s “Command Line Rust” into observe.

I’ve all the time liked working from the command line. Each grep, cat, and curl into that little black field jogs my memory numerous driving an previous automotive. It might be somewhat bit more durable to show, it would make some noises and complain, however you’re feeling a connection to the machine. And being lively with the code, very like taking notes, helps issues stick.

Not being a Rust guru, I made a decision to place Q to the take a look at. I nonetheless have loads of questions in regards to the language, idioms, the possession mannequin, and customary libraries I’d seen in pattern code, like Tokio. If I’m being sincere, studying interpret what the compiler is objecting to might be the toughest half for me of programming in Rust. With Q open in my IDE, it was simple to fireplace off “silly” questions with out stigma, and utilizing the references it offered meant that I didn’t must dig by way of troves of documentation.

Summary of Tokio

Because the CLI began to take form, Q performed a extra vital position, offering deeper insights that knowledgeable coding and design selections. As an illustration, I used to be curious whether or not utilizing slice references would introduce inefficiencies with massive lists of things. Q promptly defined that whereas slices of arrays could possibly be extra environment friendly than creating new arrays, there’s a risk of efficiency impacts at scale. It felt like a dialog – I may bounce concepts off of Q, freely ask observe up questions, and obtain rapid, non-judgmental responses.

Advice from Q on slices in Rust

The very last thing I’ll point out is the characteristic to ship code on to Q. I’ve been experimenting with code refactoring and optimization, and it has helped me construct a greater understanding of Rust, and pushed me to suppose extra critically in regards to the code I’ve written. It goes to indicate simply how vital it’s to create instruments that meet builders the place they’re already snug — in my case, the IDE.

Send code to Q

Coming quickly…

Within the subsequent few weeks, the plan is to share my code for my Rust CLI. I want a little bit of time to shine this off, and have of us with a bit extra expertise overview it, however right here’s a sneak peek:

Sneak peak of the Rust CLI

As all the time, now go construct! And get your fingers soiled whereas doing it.