Text Classification with LLMs in R - A Short Course

Why are large language transformer models (LLMs) so popular nowadays?

Large language transformer models (such as GPT-5, GPT-oss-120b, and GPT-oss-20b) have gained popularity for several reasons:

1. State-of-the-art performance: These models have achieved state-of-the-art performance on a wide range of natural language processing tasks, including language translation, text summarization, question answering, and language generation.
2. Zero-shot learning: LLMs can perform tasks for which they have not been explicitly trained, a property known as zero-shot learning. This is because they have been trained on a vast amount of diverse text data, allowing them to understand the underlying patterns and relationships in natural language.
3. Scalability: LLMs are highly scalable and can be fine-tuned for specific tasks with relatively small amounts of task-specific data.
4. General-purpose: LLMs are designed to be general-purpose, meaning they can be used for a wide variety of natural language processing tasks without the need for specialized models for each task.
5. Ease of use: Many LLMs are available as pre-trained models, allowing developers and researchers to use them without the need for extensive training or expertise in natural language processing.

Overall, the combination of state-of-the-art performance, zero-shot learning, scalability, general-purpose design, and ease of use makes large language transformer models highly attractive for a wide range of natural language processing applications.

This course is designed as a first introduction to natural language processing and large language models for research applications, covering some basic concepts and applications of transformer models in R.

This is a hands-on course with instructor-led software demonstrations and guided exercises. These guided exercises are designed for the R language, so you should use a computer with a recent version of R (version 4.1.3 or later) and RStudio (version 2022.02.1+461 or later).

To follow along with the exercises, you should have good familiarity with the use of R, including opening and executing data files and programs, as well as performing very basic data manipulation and analyses.

The exercises will also use UV, a fast Python package manager, and free API keys from LLM cloud providers. All necessary instructions will be provided before the seminar begins.

If you’d like to take this course but are concerned that you don’t know enough R, there are excellent online resources for learning the basics. Here are our recommendations.

The course is designed for participants who have a solid basic understanding of R and are interested in applying NLP techniques to extract insights from unstructured text data for research purposes.

Introduction to large language models

• • Understanding transformer architectures
  • Overview of state-of-the-art LLMs
  • Practical applications in research

Text generation for classification

• • Working with Groq API
  • Using free and open-source models
  • Implementing text generation in R

Text mining fundamentals

• • What is text mining and its common applications
  • From texts to structured data
  • Text tokenization, stop word removal, and stemming
  • Traditional bag-of-words approaches
  • Transforming text data into a usable format for modeling

Word and document embeddings using LLMs

• • Introduction to LLM embeddings and their use in text classification
  • Different types of embeddings
  • BERT and other LLM embeddings in R
  • Working with encoder models

Network-based text classification

• • Exploratory Graph Analysis (EGA) for text data
  • Networks for text classification
  • Mining embeddings with network analysis in R

Advanced classification techniques

• • Zero-shot classification with Hugging Face Transformers
  • Implementation of zero-shot methods in R
  • Retrieval-augmented generation (RAG) for text analysis
  • Automatic text classification and summarization using R

Time-series text analysis (if time allows)

• • Dynamic Exploratory Graph Analysis for repeated measures
  • Topic modeling for intensive longitudinal text data
  • Generalized local linear approximation and time-delay embedding

“I really like how Dr. Golino presented the material like a learning journey. Even with very little background on the topic, I felt like I understood what was happening and why it was important.”
  Roberta O’Malley, University of South Florida

“The presenter was extremely knowledgeable about the topics discussed. Plenty of time for questions to be asked and answered. The course material provided was well put together.” 
  Cody Knight, Department of Veterans Affairs

“The instructor is passionate and clear in his explanations. It’s great to learn from an expert who is working actively in the field. He’s able to explain complex concepts in a simplified way. I also liked the general knowledge on LLMs and other applications that I gained from the course.” 
  Catherine Chanfreau, Department of Veterans Affairs

“This course exceeded my expectations. The presentations were top-notch, providing clear insights into complex concepts. The discussions were engaging, fostering a collaborative learning environment. I highly recommend this course to anyone interested in exploring large language models.” 
  Sepideh Banava, University of California San Francisco

“I highly recommend this course to researchers looking for an introduction to using large language models in their research. It is well-structured, comprehensive, and the support provided by the instructor is second to none.”
  William Rayo, Oregon State University