Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2334 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.5201. Min Act: -0.1700

Data Index: 8111235 (Open Web Text)

Max Activating Token Index: 772

Text #1

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.5014. Min Act: -0.1700

Data Index: 5354737 (Open Web Text)

Max Activating Token Index: 328

Text #2

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.4506. Min Act: -0.1700

Data Index: 7012419 (Open Web Text)

Max Activating Token Index: 124

Text #3

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.3925. Min Act: -0.1700

Data Index: 242055 (Open Web Text)

Max Activating Token Index: 65

Text #4

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.3205. Min Act: -0.1700

Data Index: 6729474 (Open Web Text)

Max Activating Token Index: 511

Text #5

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.3190. Min Act: -0.1700

Data Index: 3784496 (Open Web Text)

Max Activating Token Index: 389

Text #6

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2689. Min Act: -0.1700

Data Index: 2001550 (Open Web Text)

Max Activating Token Index: 134

Text #7

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2463. Min Act: -0.1700

Data Index: 59479 (Open Web Text)

Max Activating Token Index: 387

Text #8

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2435. Min Act: -0.1700

Data Index: 4687005 (Open Web Text)

Max Activating Token Index: 592

Text #9

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.3043. Min Act: -0.1700

Data Index: 150860 (Open Web Text)

Max Activating Token Index: 847

Text #10

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2610. Min Act: -0.1700

Data Index: 18334 (Open Web Text)

Max Activating Token Index: 126

Text #11

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2797. Min Act: -0.1699

Data Index: 1462701 (Open Web Text)

Max Activating Token Index: 495

Text #12

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2219. Min Act: -0.1700

Data Index: 2001549 (Open Web Text)

Max Activating Token Index: 732

Text #13

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2174. Min Act: -0.1700

Data Index: 18326 (Open Web Text)

Max Activating Token Index: 709

Text #14

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2337. Min Act: -0.1700

Data Index: 6936476 (Open Web Text)

Max Activating Token Index: 561

Text #15

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2063. Min Act: -0.1700

Data Index: 5694783 (Open Web Text)

Max Activating Token Index: 122

Text #16

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2389. Min Act: -0.1700

Data Index: 8464189 (Open Web Text)

Max Activating Token Index: 632

Text #17

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.1931. Min Act: -0.1700

Data Index: 3379970 (Open Web Text)

Max Activating Token Index: 262

Text #18

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.1922. Min Act: -0.1700

Data Index: 43012 (Open Web Text)

Max Activating Token Index: 848

Text #19

Max Range: 4.5201. Min Range: -4.5201

Max Act: 4.2017. Min Act: -0.1700

Data Index: 4369932 (Open Web Text)

Max Activating Token Index: 293