Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2151 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.3318. Min Act: -0.1700

Data Index: 6150144 (Open Web Text)

Max Activating Token Index: 62

Text #1

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2982. Min Act: -0.1700

Data Index: 884007 (Open Web Text)

Max Activating Token Index: 422

Text #2

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.3133. Min Act: -0.1700

Data Index: 1320324 (Open Web Text)

Max Activating Token Index: 16

Text #3

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.3213. Min Act: -0.1700

Data Index: 5180710 (Open Web Text)

Max Activating Token Index: 436

Text #4

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.3000. Min Act: -0.1700

Data Index: 4505963 (Open Web Text)

Max Activating Token Index: 163

Text #5

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.3160. Min Act: -0.1700

Data Index: 7011059 (Open Web Text)

Max Activating Token Index: 971

Text #6

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2765. Min Act: -0.1700

Data Index: 2587355 (Open Web Text)

Max Activating Token Index: 103

Text #7

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2796. Min Act: -0.1700

Data Index: 1310243 (Open Web Text)

Max Activating Token Index: 486

Text #8

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2857. Min Act: -0.1700

Data Index: 7165889 (Open Web Text)

Max Activating Token Index: 459

Text #9

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2600. Min Act: -0.1700

Data Index: 80337 (Open Web Text)

Max Activating Token Index: 9

Text #10

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2836. Min Act: -0.1700

Data Index: 108431 (Open Web Text)

Max Activating Token Index: 180

Text #11

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2662. Min Act: -0.1700

Data Index: 6237021 (Open Web Text)

Max Activating Token Index: 322

Text #12

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2674. Min Act: -0.1700

Data Index: 3416260 (Open Web Text)

Max Activating Token Index: 33

Text #13

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.3010. Min Act: -0.1700

Data Index: 3292331 (Open Web Text)

Max Activating Token Index: 598

Text #14

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2582. Min Act: -0.1700

Data Index: 2896566 (Open Web Text)

Max Activating Token Index: 345

Text #15

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2625. Min Act: -0.1700

Data Index: 5888185 (Open Web Text)

Max Activating Token Index: 52

Text #16

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2776. Min Act: -0.1700

Data Index: 5710561 (Open Web Text)

Max Activating Token Index: 87

Text #17

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2705. Min Act: -0.1700

Data Index: 2551262 (Open Web Text)

Max Activating Token Index: 106

Text #18

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2400. Min Act: -0.1700

Data Index: 4631592 (Open Web Text)

Max Activating Token Index: 475

Text #19

Max Range: 4.3318. Min Range: -4.3318

Max Act: 4.2508. Min Act: -0.1700

Data Index: 511305 (Open Web Text)

Max Activating Token Index: 457