Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2229 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.4693. Min Act: -0.1700

Data Index: 5059356 (Open Web Text)

Max Activating Token Index: 269

Text #1

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.4543. Min Act: -0.1700

Data Index: 6968987 (Open Web Text)

Max Activating Token Index: 853

Text #2

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.4372. Min Act: -0.1700

Data Index: 855765 (Open Web Text)

Max Activating Token Index: 728

Text #3

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.4095. Min Act: -0.1700

Data Index: 7488353 (Open Web Text)

Max Activating Token Index: 408

Text #4

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.3700. Min Act: -0.1700

Data Index: 2039128 (Open Web Text)

Max Activating Token Index: 614

Text #5

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.3698. Min Act: -0.1700

Data Index: 8536288 (Open Web Text)

Max Activating Token Index: 516

Text #6

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.3364. Min Act: -0.1700

Data Index: 167935 (Open Web Text)

Max Activating Token Index: 703

Text #7

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.3403. Min Act: -0.1700

Data Index: 937853 (Open Web Text)

Max Activating Token Index: 159

Text #8

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.3128. Min Act: -0.1700

Data Index: 5995311 (Open Web Text)

Max Activating Token Index: 786

Text #9

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.3382. Min Act: -0.1700

Data Index: 6257876 (Open Web Text)

Max Activating Token Index: 724

Text #10

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.3364. Min Act: -0.1700

Data Index: 6924595 (Open Web Text)

Max Activating Token Index: 825

Text #11

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.2964. Min Act: -0.1700

Data Index: 5657552 (Open Web Text)

Max Activating Token Index: 586

Text #12

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.3148. Min Act: -0.1700

Data Index: 5913765 (Open Web Text)

Max Activating Token Index: 960

Text #13

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.3158. Min Act: -0.1700

Data Index: 3527492 (Open Web Text)

Max Activating Token Index: 553

Text #14

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.2984. Min Act: -0.1700

Data Index: 4513983 (Open Web Text)

Max Activating Token Index: 620

Text #15

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.2942. Min Act: -0.1700

Data Index: 411944 (Open Web Text)

Max Activating Token Index: 248

Text #16

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.2635. Min Act: -0.1700

Data Index: 7226173 (Open Web Text)

Max Activating Token Index: 116

Text #17

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.2765. Min Act: -0.1700

Data Index: 4250169 (Open Web Text)

Max Activating Token Index: 489

Text #18

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.2741. Min Act: -0.1700

Data Index: 2046761 (Open Web Text)

Max Activating Token Index: 481

Text #19

Max Range: 3.4693. Min Range: -3.4693

Max Act: 3.2790. Min Act: -0.1700

Data Index: 4436885 (Open Web Text)

Max Activating Token Index: 1001