Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1142 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1233. Min Range: -3.1233

Max Act: 3.1233. Min Act: -0.1700

Data Index: 5128650 (Open Web Text)

Max Activating Token Index: 819

Text #1

Max Range: 3.1233. Min Range: -3.1233

Max Act: 3.0738. Min Act: -0.1700

Data Index: 5687222 (Open Web Text)

Max Activating Token Index: 277

Text #2

Max Range: 3.1233. Min Range: -3.1233

Max Act: 3.0289. Min Act: -0.1700

Data Index: 2294419 (Open Web Text)

Max Activating Token Index: 530

Text #3

Max Range: 3.1233. Min Range: -3.1233

Max Act: 3.0382. Min Act: -0.1700

Data Index: 1390413 (Open Web Text)

Max Activating Token Index: 270

Text #4

Max Range: 3.1233. Min Range: -3.1233

Max Act: 3.0197. Min Act: -0.1700

Data Index: 8463374 (Open Web Text)

Max Activating Token Index: 982

Text #5

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9953. Min Act: -0.1700

Data Index: 7564324 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9784. Min Act: -0.1700

Data Index: 2106025 (Open Web Text)

Max Activating Token Index: 323

Text #7

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9723. Min Act: -0.1700

Data Index: 1510364 (Open Web Text)

Max Activating Token Index: 295

Text #8

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9677. Min Act: -0.1700

Data Index: 4345668 (Open Web Text)

Max Activating Token Index: 647

Text #9

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9600. Min Act: -0.1700

Data Index: 1597026 (Open Web Text)

Max Activating Token Index: 244

Text #10

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9378. Min Act: -0.1700

Data Index: 1673142 (Open Web Text)

Max Activating Token Index: 66

Text #11

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9411. Min Act: -0.1700

Data Index: 4104197 (Open Web Text)

Max Activating Token Index: 61

Text #12

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9416. Min Act: -0.1700

Data Index: 8058802 (Open Web Text)

Max Activating Token Index: 760

Text #13

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9576. Min Act: -0.1700

Data Index: 1002485 (Open Web Text)

Max Activating Token Index: 953

Text #14

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9192. Min Act: -0.1700

Data Index: 6519535 (Open Web Text)

Max Activating Token Index: 262

Text #15

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9188. Min Act: -0.1700

Data Index: 5119799 (Open Web Text)

Max Activating Token Index: 743

Text #16

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9107. Min Act: -0.1700

Data Index: 209335 (Open Web Text)

Max Activating Token Index: 65

Text #17

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9243. Min Act: -0.1700

Data Index: 8207318 (Open Web Text)

Max Activating Token Index: 467

Text #18

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9022. Min Act: -0.1700

Data Index: 4960782 (Open Web Text)

Max Activating Token Index: 52

Text #19

Max Range: 3.1233. Min Range: -3.1233

Max Act: 2.9038. Min Act: -0.1700

Data Index: 7894822 (Open Web Text)

Max Activating Token Index: 3