Cross Modal Attention Based Instruction Grounding For Visual Reasoning

via “instruction-tuned visual reasoning with in-context learning”

Salesforce's efficient vision-language bridge model.

Unique: Enables instruction-tuned visual reasoning by leveraging frozen LLM's instruction-following and in-context learning capabilities, allowing zero-shot adaptation to new reasoning tasks via prompting without fine-tuning

vs others: More flexible than task-specific VQA models because instructions enable diverse reasoning types, and more efficient than fine-tuning because in-context learning adapts to new tasks via prompts