Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1219 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.4132. Min Act: -0.1700

Data Index: 1898899 (Open Web Text)

Max Activating Token Index: 509

Text #1

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.4000. Min Act: -0.1700

Data Index: 380325 (Open Web Text)

Max Activating Token Index: 322

Text #2

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.4025. Min Act: -0.1700

Data Index: 7009360 (Open Web Text)

Max Activating Token Index: 318

Text #3

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.3729. Min Act: -0.1700

Data Index: 5859514 (Open Web Text)

Max Activating Token Index: 591

Text #4

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.3624. Min Act: -0.1700

Data Index: 3808147 (Open Web Text)

Max Activating Token Index: 116

Text #5

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.3583. Min Act: -0.1700

Data Index: 1029745 (Open Web Text)

Max Activating Token Index: 35

Text #6

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.3227. Min Act: -0.1700

Data Index: 2692003 (Open Web Text)

Max Activating Token Index: 55

Text #7

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.3324. Min Act: -0.1700

Data Index: 5290249 (Open Web Text)

Max Activating Token Index: 355

Text #8

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.3169. Min Act: -0.1700

Data Index: 8252907 (Open Web Text)

Max Activating Token Index: 28

Text #9

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.2945. Min Act: -0.1700

Data Index: 2236762 (Open Web Text)

Max Activating Token Index: 79

Text #10

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.3097. Min Act: -0.1700

Data Index: 7849779 (Open Web Text)

Max Activating Token Index: 158

Text #11

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.2542. Min Act: -0.1700

Data Index: 3603896 (Open Web Text)

Max Activating Token Index: 209

Text #12

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.2553. Min Act: -0.1700

Data Index: 4744190 (Open Web Text)

Max Activating Token Index: 863

Text #13

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.2377. Min Act: -0.1700

Data Index: 1002482 (Open Web Text)

Max Activating Token Index: 382

Text #14

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.2431. Min Act: -0.1700

Data Index: 8224859 (Open Web Text)

Max Activating Token Index: 466

Text #15

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.2022. Min Act: -0.1700

Data Index: 2789062 (Open Web Text)

Max Activating Token Index: 70

Text #16

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.2136. Min Act: -0.1700

Data Index: 5656981 (Open Web Text)

Max Activating Token Index: 259

Text #17

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.2226. Min Act: -0.1700

Data Index: 4057280 (Open Web Text)

Max Activating Token Index: 682

Text #18

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.2177. Min Act: -0.1700

Data Index: 4410963 (Open Web Text)

Max Activating Token Index: 504

Text #19

Max Range: 3.4132. Min Range: -3.4132

Max Act: 3.2108. Min Act: -0.1700

Data Index: 1930898 (Open Web Text)

Max Activating Token Index: 283