I am going to be starting a new project based on using AI as an educational resource for both faculty and students. I will create a new searchable category for this here on the blog called “AI” in case anyone is interested in filtering down to this particular journey. I intend for this to move pretty quickly so as to hopefully deploy something while it is still in fact an emergent technology.
First little mention goes to this website tool that helps one understand the currency that this will be running on via MS Azure: https://platform.openai.com/tokenizer
When running this service, you operate with a quota of tokens, and the tokens cost money.
“Tokens in the context of OpenAI GPT models are clusters of characters representing the fundamental unit of text. These tokens are generated by a tokenizer algorithm that segregates text into smaller segments following certain rules, such as spaces, punctuation marks, and special characters.”