Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1075 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0962. Min Act: -0.1700

Data Index: 6466021 (Open Web Text)

Max Activating Token Index: 993

Text #1

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0835. Min Act: -0.1700

Data Index: 8663052 (Open Web Text)

Max Activating Token Index: 726

Text #2

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0297. Min Act: -0.1700

Data Index: 6567203 (Open Web Text)

Max Activating Token Index: 327

Text #3

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0407. Min Act: -0.1700

Data Index: 8402728 (Open Web Text)

Max Activating Token Index: 357

Text #4

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0539. Min Act: -0.1700

Data Index: 3596862 (Open Web Text)

Max Activating Token Index: 346

Text #5

Max Range: 5.0962. Min Range: -5.0962

Max Act: 4.9976. Min Act: -0.1700

Data Index: 5797943 (Open Web Text)

Max Activating Token Index: 941

Text #6

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0301. Min Act: -0.1700

Data Index: 8203434 (Open Web Text)

Max Activating Token Index: 177

Text #7

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0060. Min Act: -0.1700

Data Index: 4839506 (Open Web Text)

Max Activating Token Index: 568

Text #8

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0855. Min Act: -0.1700

Data Index: 2751161 (Open Web Text)

Max Activating Token Index: 452

Text #9

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0299. Min Act: -0.1700

Data Index: 7488288 (Open Web Text)

Max Activating Token Index: 1001

Text #10

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0225. Min Act: -0.1700

Data Index: 2996965 (Open Web Text)

Max Activating Token Index: 542

Text #11

Max Range: 5.0962. Min Range: -5.0962

Max Act: 5.0094. Min Act: -0.1700

Data Index: 5876366 (Open Web Text)

Max Activating Token Index: 284

Text #12

Max Range: 5.0962. Min Range: -5.0962

Max Act: 4.9846. Min Act: -0.1700

Data Index: 5126280 (Open Web Text)

Max Activating Token Index: 903

Text #13

Max Range: 5.0962. Min Range: -5.0962

Max Act: 4.9738. Min Act: -0.1700

Data Index: 4609336 (Open Web Text)

Max Activating Token Index: 760

Text #14

Max Range: 5.0962. Min Range: -5.0962

Max Act: 4.9756. Min Act: -0.1700

Data Index: 5653265 (Open Web Text)

Max Activating Token Index: 529

Text #15

Max Range: 5.0962. Min Range: -5.0962

Max Act: 4.9944. Min Act: -0.1700

Data Index: 4182537 (Open Web Text)

Max Activating Token Index: 254

Text #16

Max Range: 5.0962. Min Range: -5.0962

Max Act: 4.9809. Min Act: -0.1700

Data Index: 2528323 (Open Web Text)

Max Activating Token Index: 758

Text #17

Max Range: 5.0962. Min Range: -5.0962

Max Act: 4.9678. Min Act: -0.1700

Data Index: 6698046 (Open Web Text)

Max Activating Token Index: 161

Text #18

Max Range: 5.0962. Min Range: -5.0962

Max Act: 4.9692. Min Act: -0.1700

Data Index: 561305 (Open Web Text)

Max Activating Token Index: 420

Text #19

Max Range: 5.0962. Min Range: -5.0962

Max Act: 4.9631. Min Act: -0.1700

Data Index: 3072269 (Open Web Text)

Max Activating Token Index: 941