Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 161 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.6661. Min Act: -0.1700

Data Index: 8563871 (Open Web Text)

Max Activating Token Index: 1016

Text #1

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.6002. Min Act: -0.1700

Data Index: 8430965 (Open Web Text)

Max Activating Token Index: 232

Text #2

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.5516. Min Act: -0.1700

Data Index: 6177891 (Open Web Text)

Max Activating Token Index: 953

Text #3

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.5471. Min Act: -0.1700

Data Index: 8688226 (Open Web Text)

Max Activating Token Index: 643

Text #4

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.5427. Min Act: -0.1700

Data Index: 3330120 (Open Web Text)

Max Activating Token Index: 1007

Text #5

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.5381. Min Act: -0.1700

Data Index: 4351026 (Open Web Text)

Max Activating Token Index: 878

Text #6

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4876. Min Act: -0.1700

Data Index: 5678198 (Open Web Text)

Max Activating Token Index: 473

Text #7

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4732. Min Act: -0.1700

Data Index: 3622372 (Open Web Text)

Max Activating Token Index: 296

Text #8

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4827. Min Act: -0.1700

Data Index: 1188415 (Open Web Text)

Max Activating Token Index: 420

Text #9

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4707. Min Act: -0.1700

Data Index: 964225 (Open Web Text)

Max Activating Token Index: 579

Text #10

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4542. Min Act: -0.1700

Data Index: 1877872 (Open Web Text)

Max Activating Token Index: 233

Text #11

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4452. Min Act: -0.1700

Data Index: 4246432 (Open Web Text)

Max Activating Token Index: 631

Text #12

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4377. Min Act: -0.1700

Data Index: 7833252 (Open Web Text)

Max Activating Token Index: 425

Text #13

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4399. Min Act: -0.1700

Data Index: 234541 (Open Web Text)

Max Activating Token Index: 346

Text #14

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4399. Min Act: -0.1700

Data Index: 4579150 (Open Web Text)

Max Activating Token Index: 774

Text #15

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4269. Min Act: -0.1700

Data Index: 4168336 (Open Web Text)

Max Activating Token Index: 670

Text #16

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4262. Min Act: -0.1700

Data Index: 6260100 (Open Web Text)

Max Activating Token Index: 73

Text #17

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4257. Min Act: -0.1700

Data Index: 3862694 (Open Web Text)

Max Activating Token Index: 255

Text #18

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4188. Min Act: -0.1700

Data Index: 2540051 (Open Web Text)

Max Activating Token Index: 719

Text #19

Max Range: 3.6661. Min Range: -3.6661

Max Act: 3.4049. Min Act: -0.1700

Data Index: 6111515 (Open Web Text)

Max Activating Token Index: 881