AI is all the rage these days and I’d be remiss if I wasn’t talking about it. Especially since I’m a bit of a proponent and use it in a number of my workflows. I’m not going to be talking about training LLMs or the newest learning algorithms like Q* (I’m not sure many could discuss that anyway). I want to provide insight on how a man-of-many-hats is using AI to help his productivity and creativity.
I’m tempted to start by saying “where can’t you use AI” because that almost seems simpler as it is starting to touch every part of our lives. We may not even realize that we’ve been using forms of AI for a long time now. Text-to-speech and Speech-to-text are widely used AI applications (who hasn’t used Siri, Alexa, or Google?), it’s been part of gaming for a long time (A* pathfinding), we’ve all seen it on the silver screen bringing actors back from the dead or de-aging live actors considerably (the newest iterations are barely distinguishable from practical effects and make up). But now we all have easy access to these tools. Generative AI became much more popular about a year ago when it got really good at making art.
Now that we all have easy access to Generative AI it is seemingly everywhere. With the recent OpenAI debacle with Sam Altman being fired and rehired faster than ChatGPT can generate a bad joke, AI is on everyone’s mind. And it’s absolutely fun and amazing! For me it has removed creative blockers so that I could get moving on projects again. I use it for fun and for business. It’s even made me hopeful for a future that often looks so lost. Let’s dive in.
GitHub Copilot
We started experimenting with this at work as a productivity enhancement tool earlier this year. Some developers liked it. Some found it frustrating similar to how many of us would be annoyed at autocorrect. It could get in the way and provide suggestions that were far from ideal. Over the past few months it has become increasingly better and has really helped improve my coding productivity.
Since I don’t do a ton of coding and I often use different languages it can be challenging for me to remember syntax and language features. At work I use Ruby and Python primarily while at home I use JavaScript/Node, C#/Unity3D, and Flutter. Too many languages for my 45-year-old brain to remember perfectly. I find it pretty easy to read these languages when doing code reviews but when it comes time to generate code of my own I find that I’m spending a lot of time looking through documentation to re-re-re-discover how to do something. Not only does that eat up precious production time but it also is demoralizing when you look up how to map a Ruby array for the 20th time because you’re frequently interrupted and all you can recall is how to do it in JS because that’s where you learned it. It’s akin to trying to learn a second language but every sentence you say in that language you’re interrupted and have to talk in your primary language. Heavy context switches. It’s the life of an engineering manager, I suppose.
Copilot has been a game changer for me. I can work on some code for 30 minutes one day then not touch it for a week and come back to the context within a few minutes because I can just ask the AI how to do X instead of fumble through documentation and conflicting StackOverflow answers. I found it useful enough at work that I got my own subscription for home usage in hopes that it would unblock me on personal projects. And it has!
I’m happy to announce that I’m now working on the rewrite of Wordspionage (a spy-themed word game with cool special moves), my first “big” game. It’s been off the app stores for years now but we have a group of die-hard players that I’d love to give an updated version to. I wrote the app when I knew nothing about software engineering and I ended up re-inventing the wheel in a few places. One place was that I made my own REST API in PHP before I even had a clue what a REST API was (hey I’d only been developing software for a year and completely self-taught – gimme a break). So that’s been rewritten with a websockets REST API that’s more appropriate for a game even if it is a turn-based game. I’d also taken a long time off of Unity3D development. I hadn’t touched Unity since my last job 2 years ago and when I was there I didn’t have much time to code as I was managing people and projects all the time. I did get a sporadic chance to vet new tech but that’s far from being a daily contributor. Copilot has completely unblocked me. I’m fixing things that I was having trouble with before with so much ease.
A trivial problem I was having was that there’s a tile tray for the player’s letters to be in when they aren’t played and when you drop a tile back into the tray from the game board they would sometimes overlap or not move out of the way fluidly. There was an odd logic block that I couldn’t get past. I had some code there and I would write comments / pseudo code about how it should work. Copilot got it and wrote the fixes for me. It enabled me to iterate more quickly and get quirky things like Visual Studio’s Unity debugger working well (I would just ask it the issue and it would give a more relevant answer than I could find in 4x the time on the Googles). I’ve been flying through tasks ever since. There is a sticky algorithm for the wheel spin in the game that I’m excited to see how it will handle. The wheel has 8 sections that you can land on but many sections it can only land on once so the random selection is done before the player spins. There’s a bit of a challenge around making the wheel spin look natural and slow down smoothly. I solved it before but that was in a different language and a version of myself that was 10 years younger. So I’m curious about how it’ll pan out this time with Copilot’s help.
Copilot has improved my productivity and made me actually interested in writing code again as it has removed a lot of my frustrations. I’m actually a little regretful that I’m taking time to write this article right now instead of working on the game. I can’t wait to bring this much needed update to our die hard players and hopefully bring in new players to the mix. Very exciting!
Dungeons & Dragons
I’m not sure how many people are using generative AI for D&D content but me and one of my friends are and it’s friggin’ great! Earlier this year I ran an Easter-themed game for Easter that leaned heavily on AI to help generate it. I definitely couldn’t take the content directly from ChatGPT because it didn’t always quite fit but it provided me with the building blocks and inspiration. I used it to generate some of the narrative descriptions (I did have to reword some things but not a lot), I used it to generate the imagery and NPC token images, I used it to generate stat blocks for the NPCs, and items to reward the players with. I couldn’t use it for making a map, though. It really wants to make isometric perspective maps and doesn’t like doing top-down even when told repeatedly. This was primarily done with ChatGPT and DALL-E 2 but some Midjourney too. The adventure outline can be found here but it’s not well-polished. I was able to create that game in a couple of hours. Without the help of generative AI it would have not only taken a lot longer but I wouldn’t have been able to find images for the weird creatures in my imagination. This was only the beginning for me.
I’ve used it for a number of one-shot games now as well as for my own character tokens. I’ve used it to come up with bad jokes, goofy poems (ask me about the love poem for the wooly mammoth…), and even riddles. It’s made writing D&D adventures so much quicker. There’s no stumbling around with creative blocks anymore for me. I can have a pretty vague theme and just start talking to the AI about it. An hour or two later I’m ready with a game.
A few of us on our D&D Discord like to post gifs and AI-generated images during adventures as a way to do a weird sort of visual storytelling that has elevated our fun. We’re posting gifs and images about the adventure and it ends up telling a unique story that is hard to interpret if you weren’t there but for those of us that were it is hilarious. I do wish it were more accessible to those who weren’t present for the gaming session but it’s brought a lot of joy to us who were playing. Maybe someday I’ll record the audio for the D&D sessions and have AI transcribe then summarize it and see if it can be used as an audio narration with a slideshow of images that we posted. That would be an amazing new storytelling form. Many folks have started streaming their D&D sessions but this is something completely different!
A tool that I’d love to have time to develop is an AI app that helps people create their own D&D adventures. ChatGPT is surprisingly good at generating D&D content and I think this would be a lot of fun for people. I do worry a little for the artists out there but to be honest, I rarely use paid art content. I have bought some but it’s always missing things I want and if I want something special it costs way too much. I have commissioned art before but it’s often hard to communicate exactly what you want to the artist. It’s a mixture of not knowing what you want and artists using creative license when you don’t want them to. AI doesn’t generate exactly what you want either. I think the fact that it’s incapable of taking exact direction removes that expectation and annoyance when it’s not following directions. To a degree, of course. It’s getting better and it’s so much faster and easier to iterate on than a real artist. But the quality is a wide gap. I do worry about our artist friends out there and how they learn to leverage AI tools to help them speed up their own processes to be able to produce more easily and less expensive.
Video Content Creation
Another path I explored was creating videos using the various AI tools out there. I tested a few and the one that really stuck out was Pictory.ai. I took top news articles of the day for AI then used ChatGPT to help create scripts for the video. The scripts required some editing but not much. It did a great job of summarizing articles and didn’t make stuff up. The output format would be inconsistent even with explicit instructions which made me concerned about making this more automated with API calls. Then I uploaded the scripts into Pictory. Getting the video to flow well was a challenge. They offer voice text to speech generation which has good voice quality. However, there’s no way to indicate longer pauses so the voice sounds like it is not taking a breath. This turns out quite unnatural. There are ways to add padding to each clip (which is generated per paragraph). But the interface to do that is not great UX. Markdown in the script would be much easier. The video clips it chooses for the content are good quality and appropriate for the subject matter. I only had to manually replace a few due to what appeared to be glitches.
The resulting videos were pretty good but too unnatural to be something I’d personally want to watch. They were much better than many of the AI-generated YouTube videos I’ve seen. Still the lack of pauses were too much. There’s just no way to convey any personality. The videos took 1-3 hours each to generate. If I got my workflow down I could see making 3-4 of these per day. This could potentially be a revenue generator for someone but I think you need 4,000 watch hours on YouTube just to start earning. At 5-minutes per video that would take a long time unless I hit the viral video lottery. Not something I have time to dedicate to. However, if the voices were better and I could add a little emotion and proper pauses then maybe.
Check out the videos below. They aren’t bad (they’re not that good either but… still early tech):
I’d like to try this again in about a year and see how the tech has progressed. I imagine we’ll see a lot more AI-generated video content in 2024. So far I’ve not seen any I’ve liked but who knows what the next year will bring?
AI Ventures
A friend and business associate that I developed Vuzop, a VR process and data visualization app, with has kept in touch through the years and we have both been bitten by the AI bug. We’ve both experienced grand claims of tech throughout the years and often would explore some related ideas only to find out we were, yet again, ahead of our time. What we wanted to do wasn’t quite possible unless you had a bunch of money to throw away. I think we’d both prefer that other, deeper, pockets flesh out bleeding edge tech before we dive in too hard. We were pretty early on the VR forefront and still haven’t seen anything like what we made. We were developing Vuzop on the Oculus DK2 back in 2014 or 2015. So when we started seeing AI become more viable for a variety of applications we got pretty excited.
The fun thing is that there are a billion possibilities with AI right now. What I present in this article is a short list geared more toward light technologists or hobbyists. In the corporate world there’s so much to wrap your head around right now that it feels like if you’re not embracing AI and integrating it into your product then you’re going to be quickly left in the dust. Similarly if you’re not using to help develop your product. Soon it will be like a second pair of hands, a contract employee that comes to help periodically, or the analyst who can cobble together a bunch of data in minutes. It’s mind-boggling how much potential these tools have to impact businesses. It’s equally as difficult to narrow down what you want to do with it.
So, we’re thinking of creating a consultancy that helps businesses navigate this landscape. We both have a passion for improving processes. We both have a vision that AI can empower businesses to do more and allow people to work less at the busy work. Our targets aren’t well-defined yet but I have a feeling that our combined expertise can be used to help small businesses become more efficient by embracing AI.
Fun Project Ideas
AI has broken through a creative block I’ve had for a while. Now I seem to be brimming with ideas. Since I’m only one person with limited capacity to deliver these concepts – here’s some! If you execute on any of these, let me know. I’d love to see them become a reality:
- D&D Adventure Creator: Generate a story structure, NPCs, monster stat blocks, token images, maps, and establishing shots. I’d love to have time for this and think that it could be a big hit with DMs.
- Personalized Therapist / Life Coach: Character.ai has a therapy bot that’s not that bad. It didn’t provide any of the needed emotional human connection that is necessary but this gap may be reduced by it knowing your history well. Use a RAG LLM model with your personal journals or set it up so that it knows how to interview you to get to know a you. I’m curious as to how this will evolve in the near future.
- AI Image feedback loop: Take an image and send it to an LLM to describe. Then take that description and feed it into an image generator. Repeat n number of times and display the output in a legible way. I did this for about 10 times and it was interesting to see the results. The artwork seemed to progress through artistic periods like classical, impressionist, then getting more abstract and modern. What does that mean?
- Bot makes Bot: GPT Bot that creates code for a Discord bot. Build out your app using a guided wizard. Can it connect all the parts together?
To The Future!
What does the future hold for us now that AI has seemingly hit critical mass? I’m not really certain but I can dream. I’m hoping AI will accelerate technological advances so that we literally don’t have to work anymore. In the next 5 years I think we’ll see the necessity for universal basic income because we’ve replaced so many jobs with robots and AI. This is what we’ve been striving towards. To make a better life for everyone including ourselves. It’ll be a major cognitive shift for many people. I’m hopeful, though. In my dreams AI takes over everything and treats us like we treat cats. Feed us, shelter us, give us toys, and dote over us. I can accept that life. Happy 2024!
