Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1567 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.3390. Min Act: -0.1700

Data Index: 7672143 (Open Web Text)

Max Activating Token Index: 186

Text #1

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.1815. Min Act: -0.1700

Data Index: 5894466 (Open Web Text)

Max Activating Token Index: 635

Text #2

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.1229. Min Act: -0.1700

Data Index: 4729016 (Open Web Text)

Max Activating Token Index: 517

Text #3

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.1048. Min Act: -0.1700

Data Index: 2596189 (Open Web Text)

Max Activating Token Index: 812

Text #4

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.1019. Min Act: -0.1700

Data Index: 2374417 (Open Web Text)

Max Activating Token Index: 321

Text #5

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.1066. Min Act: -0.1700

Data Index: 4374964 (Open Web Text)

Max Activating Token Index: 233

Text #6

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0944. Min Act: -0.1700

Data Index: 141397 (Open Web Text)

Max Activating Token Index: 305

Text #7

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0896. Min Act: -0.1700

Data Index: 8764645 (Open Web Text)

Max Activating Token Index: 603

Text #8

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0783. Min Act: -0.1700

Data Index: 6127121 (Open Web Text)

Max Activating Token Index: 686

Text #9

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.1214. Min Act: -0.1700

Data Index: 7471020 (Open Web Text)

Max Activating Token Index: 474

Text #10

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0819. Min Act: -0.1700

Data Index: 4663047 (Open Web Text)

Max Activating Token Index: 403

Text #11

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0693. Min Act: -0.1700

Data Index: 4642312 (Open Web Text)

Max Activating Token Index: 432

Text #12

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0782. Min Act: -0.1700

Data Index: 8679801 (Open Web Text)

Max Activating Token Index: 10

Text #13

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0615. Min Act: -0.1700

Data Index: 3723512 (Open Web Text)

Max Activating Token Index: 384

Text #14

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0625. Min Act: -0.1700

Data Index: 4073542 (Open Web Text)

Max Activating Token Index: 328

Text #15

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0636. Min Act: -0.1700

Data Index: 3056268 (Open Web Text)

Max Activating Token Index: 200

Text #16

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0562. Min Act: -0.1700

Data Index: 8385756 (Open Web Text)

Max Activating Token Index: 664

Text #17

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0530. Min Act: -0.1700

Data Index: 6127119 (Open Web Text)

Max Activating Token Index: 650

Text #18

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0559. Min Act: -0.1700

Data Index: 8637408 (Open Web Text)

Max Activating Token Index: 158

Text #19

Max Range: 2.3390. Min Range: -2.3390

Max Act: 2.0633. Min Act: -0.1700

Data Index: 4749771 (Open Web Text)

Max Activating Token Index: 408