Instruction Following Conversation With Extended Context Window

via “instruction-following conversation with extended context window”

The preview GPT-4 model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Training data: up to Dec 2023. **Note:** heavily rate limited by OpenAI while...

Unique: 128K context window with improved instruction-following through reinforcement learning from human feedback (RLHF) training, enabling coherent reasoning across entire documents without context loss — achieved through sparse attention patterns and hierarchical token processing rather than full quadratic attention

vs others: Larger context window than GPT-3.5 Turbo (4K) and comparable to Claude 2 (100K), but with faster inference latency and lower per-token cost for instruction-following tasks