Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2176 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.9453. Min Act: -0.1700

Data Index: 7999864 (Open Web Text)

Max Activating Token Index: 156

Text #1

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.9543. Min Act: -0.1700

Data Index: 2476902 (Open Web Text)

Max Activating Token Index: 533

Text #2

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8959. Min Act: -0.1700

Data Index: 1238302 (Open Web Text)

Max Activating Token Index: 1013

Text #3

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8510. Min Act: -0.1700

Data Index: 7921962 (Open Web Text)

Max Activating Token Index: 280

Text #4

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8913. Min Act: -0.1700

Data Index: 5641572 (Open Web Text)

Max Activating Token Index: 46

Text #5

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8844. Min Act: -0.1700

Data Index: 5170152 (Open Web Text)

Max Activating Token Index: 691

Text #6

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8673. Min Act: -0.1700

Data Index: 3372325 (Open Web Text)

Max Activating Token Index: 916

Text #7

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8267. Min Act: -0.1700

Data Index: 7833487 (Open Web Text)

Max Activating Token Index: 876

Text #8

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.7952. Min Act: -0.1700

Data Index: 6892833 (Open Web Text)

Max Activating Token Index: 875

Text #9

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8046. Min Act: -0.1700

Data Index: 3226608 (Open Web Text)

Max Activating Token Index: 51

Text #10

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8202. Min Act: -0.1700

Data Index: 5921573 (Open Web Text)

Max Activating Token Index: 884

Text #11

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8174. Min Act: -0.1700

Data Index: 92361 (Open Web Text)

Max Activating Token Index: 220

Text #12

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8187. Min Act: -0.1700

Data Index: 4997611 (Open Web Text)

Max Activating Token Index: 525

Text #13

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8160. Min Act: -0.1700

Data Index: 4751288 (Open Web Text)

Max Activating Token Index: 98

Text #14

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8388. Min Act: -0.1700

Data Index: 7373146 (Open Web Text)

Max Activating Token Index: 259

Text #15

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.7787. Min Act: -0.1700

Data Index: 5068256 (Open Web Text)

Max Activating Token Index: 473

Text #16

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8182. Min Act: -0.1700

Data Index: 3060359 (Open Web Text)

Max Activating Token Index: 749

Text #17

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.8062. Min Act: -0.1700

Data Index: 1874690 (Open Web Text)

Max Activating Token Index: 750

Text #18

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.7805. Min Act: -0.1700

Data Index: 4437333 (Open Web Text)

Max Activating Token Index: 226

Text #19

Max Range: 3.9543. Min Range: -3.9543

Max Act: 3.7728. Min Act: -0.1700

Data Index: 6332333 (Open Web Text)

Max Activating Token Index: 41