Managing attention with roles
Posted March 3, 2023 by Alin Osan ‐ 1 min read
Getting the Transformers to do their best.
Asking a GPT model to perform a specific role activates the attention mechanism. It focuses the model's attention on the relevant parts of the input request. The model uses this focused attention to generate an adjusted response relevant to the given task. The attention mechanism is a critical component of the GPT architecture, as it enables the model to attend selectively to different parts of the input sequence based on the context and the task at hand.
I'm doing most of my research for work using expert role I created myself. I'm using less and less Google Search, still use Google, but mostly for checking factuals. Actively encouraging my team to try ChatGPT before checking Google for anything requiring analysis. Copy-paste of expert roles for other people.