Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3008 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.2466. Min Act: -0.1700

Data Index: 6120622 (Open Web Text)

Max Activating Token Index: 419

Text #1

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.2196. Min Act: -0.1700

Data Index: 4845700 (Open Web Text)

Max Activating Token Index: 391

Text #2

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.1629. Min Act: -0.1700

Data Index: 5722819 (Open Web Text)

Max Activating Token Index: 242

Text #3

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.1286. Min Act: -0.1700

Data Index: 635357 (Open Web Text)

Max Activating Token Index: 998

Text #4

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.1672. Min Act: -0.1700

Data Index: 6488017 (Open Web Text)

Max Activating Token Index: 343

Text #5

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.1021. Min Act: -0.1700

Data Index: 841620 (Open Web Text)

Max Activating Token Index: 990

Text #6

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0938. Min Act: -0.1700

Data Index: 1201916 (Open Web Text)

Max Activating Token Index: 295

Text #7

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0718. Min Act: -0.1700

Data Index: 8017082 (Open Web Text)

Max Activating Token Index: 485

Text #8

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0817. Min Act: -0.1700

Data Index: 3093254 (Open Web Text)

Max Activating Token Index: 416

Text #9

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0888. Min Act: -0.1700

Data Index: 1145899 (Open Web Text)

Max Activating Token Index: 1010

Text #10

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0551. Min Act: -0.1700

Data Index: 2450342 (Open Web Text)

Max Activating Token Index: 177

Text #11

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0629. Min Act: -0.1700

Data Index: 521408 (Open Web Text)

Max Activating Token Index: 961

Text #12

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0442. Min Act: -0.1700

Data Index: 1115516 (Open Web Text)

Max Activating Token Index: 672

Text #13

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0222. Min Act: -0.1700

Data Index: 8606308 (Open Web Text)

Max Activating Token Index: 536

Text #14

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0420. Min Act: -0.1700

Data Index: 2788952 (Open Web Text)

Max Activating Token Index: 913

Text #15

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0045. Min Act: -0.1700

Data Index: 1726739 (Open Web Text)

Max Activating Token Index: 563

Text #16

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0148. Min Act: -0.1700

Data Index: 5988362 (Open Web Text)

Max Activating Token Index: 441

Text #17

Max Range: 3.2466. Min Range: -3.2466

Max Act: 3.0062. Min Act: -0.1700

Data Index: 3162583 (Open Web Text)

Max Activating Token Index: 600

Text #18

Max Range: 3.2466. Min Range: -3.2466

Max Act: 2.9926. Min Act: -0.1700

Data Index: 479219 (Open Web Text)

Max Activating Token Index: 977

Text #19

Max Range: 3.2466. Min Range: -3.2466

Max Act: 2.9902. Min Act: -0.1700

Data Index: 6689073 (Open Web Text)

Max Activating Token Index: 391