Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2210 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0406. Min Act: -0.1700

Data Index: 7572509 (Open Web Text)

Max Activating Token Index: 7

Text #1

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0338. Min Act: -0.1700

Data Index: 2584656 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0357. Min Act: -0.1700

Data Index: 7629931 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0324. Min Act: -0.1700

Data Index: 2064150 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0280. Min Act: -0.1700

Data Index: 6634054 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0252. Min Act: -0.1700

Data Index: 2968142 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0307. Min Act: -0.1700

Data Index: 3023462 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0114. Min Act: -0.1700

Data Index: 4760097 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0183. Min Act: -0.1700

Data Index: 8017690 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0193. Min Act: -0.1700

Data Index: 7840657 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0151. Min Act: -0.1700

Data Index: 4857214 (Open Web Text)

Max Activating Token Index: 3

Text #11

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0174. Min Act: -0.1700

Data Index: 683964 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0094. Min Act: -0.1700

Data Index: 2900952 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0130. Min Act: -0.1700

Data Index: 5382458 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0111. Min Act: -0.1700

Data Index: 2985814 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0098. Min Act: -0.1700

Data Index: 7789219 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 1.0406. Min Range: -1.0406

Max Act: 0.9973. Min Act: -0.1700

Data Index: 1886628 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0041. Min Act: -0.1700

Data Index: 2685642 (Open Web Text)

Max Activating Token Index: 2

Text #18

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0026. Min Act: -0.1700

Data Index: 7296480 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 1.0406. Min Range: -1.0406

Max Act: 1.0019. Min Act: -0.1700

Data Index: 4928556 (Open Web Text)

Max Activating Token Index: 7