the-spot.net

This person does not exist. This is an AI-generated face, from https://thispersondoesnotexist.com/. And this is the face I'm using to represent [[Oracle]].

Back in November 2019, I had an idea to turn [[Oracle]] into a JARVIS-like assistant for doing my work-related, repetitive tasks, but I wanted to make it a natural language interface, like...well, JARVIS.

I realize that there is a huge difference in programming an AI bot and conversing with it, switching contexts of work, play, and daily needs... vs scripting Paul Betany to read as a disembodied voice actor. And in doing so I also realized that there are probably hundreds of people working on the Google Assistant as their day-in-day-out jobs.

I did manage to get a near fully functional work interaction programmed, asking her to "deploy <some version> to <some environment> <some type> servers"... some examples:

• Update Dev Web Servers to 4.2.1.3
• Deploy 4.0.1.2 on the dev UI servers
• ...and variations of that

...where [[Oracle]] can parse out the action (deploy, update) and the environment (dev, QA, production, demo), and the server types (web, app, database, edge) to a specific version (1.2.3.4 style) no matter what order you put it in. And that's all well and good... getting her to recognize what I was asking in context turned out to be the easier part... fulfilling the request turned out to be the ethically & technically tricky part, since I was asking her to do a work-related task, on a non-work-hosted server.

So, instead of going further down that path, I turned the natural language part towards Discord for a test run of generating an omnipresent bot in the channel. She'll take @-mentions, and depending on the statement, reply accordingly...and for specific users (like me) she'll recognize additional statements (like the deploy script).

Recently it has gotten me thinking though, if we can do more with the tsnGuild site, and do more with the Discord bot, as daunting of a task as it is, I would like to fully integrate [[Oracle]] with tsn, discord, twitch, slack, twitter, facebook... everything that I can... and then:

• Allow users to come to tsn and set up their desired notifications & channels from [[Oracle]]
• Allow users to attach their social media & chat services to their tsn account so [[Oracle]] knows who is who

And then the possibilities are endless...

• You want to be notified on Twitter when a specific tsn member goes live on Twitch, done.
• You want [[Oracle]] to notify your social media when you go live on Twitch, done.
• You want to know when someone posts, replies, messages you on tsn? No problem...where do you want it?
• You want to send an instant message to someone directly, but don't know what services they actually pay attention to? [[Oracle]] does.
• ...and literally anything else that the APIs will let you do once you know who is making the request, and who they are on the specific service.

I tried to get started on a Twitch Extension last night, and immediately got overwhelmed with it, because I have no idea what Twitch Extensions do. So I scaled it back to a registering a Twitch App that does a website integration, since I'll probably be writing an API to do these types of things, that is triggered from the natural language processor to send the relevant pieces of information to the right services.

It feels quite complicated, and it feels quite burdensome... so much so I already just want to go play video games instead of thinking about it after just writing this down. But I am curious about anyone else's thoughts... and even any other cool integrations/ideas that might be possible.

Brief update here... the Oracle Assistant is available in the tsn Discord server, and I've started a Slack version, but I haven't gotten anywhere with it. Even some of the Discord functionality has broken because some of the APIs have changed, and I haven't bothered to develop against it.

But we do have this "Chat Bot" thing at work that is honestly pretty terrible, and its failures have made me consider what would one expect a chat-bot or virtual assistant to do to help with work. I've also got this consideration of Google Assistant + Discord + Slack bots as separate services and separate Oracle instances all running on the same server (which is now a Raspberry Pi 4 on my desk)...and what if they could communication between each other.

It turns out that this might end up doing for me what tsn originally did. With tsn, I learned php, web design, database administration and basic server administration. That was enough to propel me through the last 20 years of work after just a few up-front years of work (though it was around the clock, late nights, and shirked actual-work responsibilities during my hourly jobs that didn't fit my desired career path). With [[Oracle]] I have the opportunity to learn nodejs servers, ESB communications between the bots, natural language processing & fulfillment, and even React for some front-end capabilities on Node.

I am reluctant to do any work on it related to my current job, even though I know I could deliver a better integration than the current bot we have, and better use-cases, because then I'd have to turn over the bot work to the company. But, maybe I could build out some use-cases on the side where it uses the 3rd party API, as bad as it is, for a proof of concept and then sell some portion of that integration capabilities back to the company.

All just thoughts, as I try to figure out where my career path is headed, since 20 years is a long time to coast on some forum reverse engineering and webpage development in an ever-changing landscape.

Welp, it looks like Discord has decided to move Bots away from message processing and into slash-commands. Which means all message processing is disabled for their bots, and I'll be required to use slash-commands. This really feels like a step backward from what was a "natural language" bot to a "command-driven" bot.

I guess it'll be ok, since I've already learned how to do what I wanted to do with that, but I still wish i could have a fallback to natural language processing. Instead I'll just use this required rewrite to learn TypeScript...after all, the point of tsn, Oracle, and thepizzy is to learn new things in a practical way.

So while I did finish putting in the tsn9 extension an ability to fire an event when someone posts on the forums (like this one, where I'll go check it fires for a new post) I have to rewrite the Oracle side to pull messages out of the queue of events and post them to places (Discord, Slack... twitter? facebook? 1-1 messages?).

Welp, it looks like I won't be learning TypeScript this time. Instead it was just a change in how they construct user mentions where they removed an exclamation mark from the hidden text. Now Oracle is back up and running, and pulling events from the forums, and displaying them in discord.

After a couple more tests this thing should be ready to go.

Work continues on Oracle, who will soon be called [[Orycle]] everywhere.

I just released a rewrite that started last May to move from CommonJS to ES6 so that I could upgrade some of the packages to use them how they were intended to be used. But there was also a need to combine some logic that needs to be shared across services (e.g. Discord & Slack) and to consolidate the logic into a single processor/server as the single point of communication to external sources (e.g. tsnForums, APIs, etc.).

I finally finished that work to get the new architecture in place, with a MongoDB backend, and local RabbitMQ setup on a new Raspberry Pi 5 with 8GB of memory. The previous codebase was running on a Pi 4 with 4GB. The latest code is only connected to the tsnForums and discord, but connected it to slack should be trivial now that the heavy lift of defining the plugins and functionality is done.

Not all plugins have been replicated though, only: twitch watched channels, twitch watched tags, weather alerts, and tsnForums, with a fallback to Google DialogFlow to handle some limited natural language processing.

It's that Google DialogFlow, though, that I want to replace. Back when I started this rewrite, I was also going to connect it to the "PaLM API" to handle LLMs and use the generative AI tech that was just breaking into public mainstream. But everything moved so fast once the floodgates opened, I couldn't keep up and set it all aside this winter to get a better handle on everything.

Today, though, I start the research into building with the latest iterations of that tech and bringing Generative AI to Orycle to make it more useful. And now that it's got more powerful hardware, I should be able to start working on the real functionality driving it's ORYCLE name soon enough.

It's time for another update.

Back in Feb, I was looking into adding Google's Gemini into the project. It turned out to be stupid easy by the time I got around to it. When I had started the rewrite referencing the PaLM API, it looked to be quite involved. Now after several months of iteration, it was a matter of a few simple calls to some methods after configuring the client. I was mildly disappointed that it was so simplified and there wasn't really anything to learn from it.

I spent the next couple months working on learning the Music Integration plugin that I had found. Another developer that I had starred their integration project a while back had recently committed a bunch of changes to their repo to wipe all the files in the latest releases & branches, because they decided to turn their plugin into a paid Bot as a Service. So it was just a matter of rolling back through the previous commits, to find the one right before they started wiping.

I found the last supposedly-working source and cloned the project at that point to use as reference for how they integrated their chosen plugins to discord, and to learn how this one works where my last one did not. It took a couple days of reading, chewing, and contemplating what was going on, and then I got started.

I ended up dropping some features I knew I wouldn't use, changing the data models to what I considered to be a better data structure that better utilized the capabilities of MongoDB, and cleaning up a ton of in-code filtering & sorting stuff that should have been handled by the database. I rewrote all the features, commands, and processing into the Orycle framework as a Discord-only client/server plugin.

During this process I also learned how to use the slash commands and how Interactions work, which required a change to the Orycle framework to support those things separately from the parsed text commands I had been using to this point. This let me add an unexpected feature to [[Orycle]]: the return of Quick Quips as a custom status on [[Orycle]].

This was all wrapped up a couple days ago, and posted as a paid feature on the tsnDiscord.

As I was wrapping that up, Google announced they would start charging in early May, so I started working out how I would count token usage, trend it over time, and make sure we didn't go over the usage on the free plan for now. By the time I got everything figured out, they had added a new model, which came with much higher usage limit. So I have adjusted to use that new model, and just finished testing it this morning to release it as v1.2.

Next up will be the web client to get some insights into what is happening on the server now that there is a database involved and usage metrics to monitor. This should be a big stretch of development as I have to re-learn how to start up a new vue site, but at least I have the original and the securetemps code to reference. What I don't have from those are database integrations (ST uses an api driven by java on another server, so I will probably have to build an API out of nodejs/express).

So, on to the designs, ticket creation, and research...