Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1255 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.5319. Min Act: -0.1700

Data Index: 8762411 (Open Web Text)

Max Activating Token Index: 970

Text #1

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.4036. Min Act: -0.1700

Data Index: 2391042 (Open Web Text)

Max Activating Token Index: 608

Text #2

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.4862. Min Act: -0.1700

Data Index: 2797504 (Open Web Text)

Max Activating Token Index: 867

Text #3

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.4302. Min Act: -0.1700

Data Index: 1002443 (Open Web Text)

Max Activating Token Index: 45

Text #4

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3881. Min Act: -0.1700

Data Index: 1157544 (Open Web Text)

Max Activating Token Index: 522

Text #5

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3153. Min Act: -0.1700

Data Index: 7740398 (Open Web Text)

Max Activating Token Index: 138

Text #6

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3610. Min Act: -0.1700

Data Index: 6804410 (Open Web Text)

Max Activating Token Index: 65

Text #7

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2885. Min Act: -0.1700

Data Index: 657424 (Open Web Text)

Max Activating Token Index: 753

Text #8

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2997. Min Act: -0.1700

Data Index: 3870416 (Open Web Text)

Max Activating Token Index: 125

Text #9

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3046. Min Act: -0.1700

Data Index: 2816859 (Open Web Text)

Max Activating Token Index: 94

Text #10

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3057. Min Act: -0.1700

Data Index: 7008002 (Open Web Text)

Max Activating Token Index: 713

Text #11

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.3023. Min Act: -0.1700

Data Index: 1494633 (Open Web Text)

Max Activating Token Index: 270

Text #12

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2873. Min Act: -0.1700

Data Index: 6246724 (Open Web Text)

Max Activating Token Index: 86

Text #13

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2478. Min Act: -0.1700

Data Index: 4883105 (Open Web Text)

Max Activating Token Index: 128

Text #14

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2625. Min Act: -0.1700

Data Index: 3740121 (Open Web Text)

Max Activating Token Index: 762

Text #15

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2224. Min Act: -0.1700

Data Index: 3392161 (Open Web Text)

Max Activating Token Index: 95

Text #16

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2712. Min Act: -0.1700

Data Index: 5354402 (Open Web Text)

Max Activating Token Index: 298

Text #17

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2663. Min Act: -0.1700

Data Index: 7441938 (Open Web Text)

Max Activating Token Index: 782

Text #18

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2646. Min Act: -0.1700

Data Index: 2869648 (Open Web Text)

Max Activating Token Index: 949

Text #19

Max Range: 3.5319. Min Range: -3.5319

Max Act: 3.2921. Min Act: -0.1700

Data Index: 3362086 (Open Web Text)

Max Activating Token Index: 560