Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1043 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.9529. Min Act: -0.1698

Data Index: 6139766 (Open Web Text)

Max Activating Token Index: 582

Text #1

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.9356. Min Act: -0.1700

Data Index: 1351732 (Open Web Text)

Max Activating Token Index: 456

Text #2

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8763. Min Act: -0.1696

Data Index: 155719 (Open Web Text)

Max Activating Token Index: 307

Text #3

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8844. Min Act: -0.1700

Data Index: 5429883 (Open Web Text)

Max Activating Token Index: 861

Text #4

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8487. Min Act: -0.1700

Data Index: 3297775 (Open Web Text)

Max Activating Token Index: 877

Text #5

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8590. Min Act: -0.1698

Data Index: 390519 (Open Web Text)

Max Activating Token Index: 561

Text #6

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8573. Min Act: -0.1700

Data Index: 2088345 (Open Web Text)

Max Activating Token Index: 529

Text #7

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8801. Min Act: -0.1700

Data Index: 524079 (Open Web Text)

Max Activating Token Index: 398

Text #8

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8517. Min Act: -0.1699

Data Index: 2758462 (Open Web Text)

Max Activating Token Index: 395

Text #9

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8611. Min Act: -0.1700

Data Index: 218476 (Open Web Text)

Max Activating Token Index: 467

Text #10

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8342. Min Act: -0.1691

Data Index: 7158905 (Open Web Text)

Max Activating Token Index: 428

Text #11

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8366. Min Act: -0.1700

Data Index: 2419368 (Open Web Text)

Max Activating Token Index: 494

Text #12

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8305. Min Act: -0.1699

Data Index: 8025318 (Open Web Text)

Max Activating Token Index: 523

Text #13

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8260. Min Act: -0.1700

Data Index: 3132596 (Open Web Text)

Max Activating Token Index: 667

Text #14

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.7990. Min Act: -0.1700

Data Index: 7386991 (Open Web Text)

Max Activating Token Index: 813

Text #15

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8179. Min Act: -0.1700

Data Index: 1529161 (Open Web Text)

Max Activating Token Index: 555

Text #16

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8224. Min Act: -0.1700

Data Index: 2044868 (Open Web Text)

Max Activating Token Index: 552

Text #17

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8089. Min Act: -0.1700

Data Index: 686818 (Open Web Text)

Max Activating Token Index: 421

Text #18

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.7985. Min Act: -0.1698

Data Index: 676992 (Open Web Text)

Max Activating Token Index: 667

Text #19

Max Range: 1.9529. Min Range: -1.9529

Max Act: 1.8153. Min Act: -0.1700

Data Index: 5241850 (Open Web Text)

Max Activating Token Index: 752