Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 397 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1392. Min Range: -3.1392

Max Act: 3.1392. Min Act: -0.1700

Data Index: 4109022 (Open Web Text)

Max Activating Token Index: 113

Text #1

Max Range: 3.1392. Min Range: -3.1392

Max Act: 3.0774. Min Act: -0.1700

Data Index: 8166823 (Open Web Text)

Max Activating Token Index: 772

Text #2

Max Range: 3.1392. Min Range: -3.1392

Max Act: 3.0852. Min Act: -0.1700

Data Index: 1295897 (Open Web Text)

Max Activating Token Index: 308

Text #3

Max Range: 3.1392. Min Range: -3.1392

Max Act: 3.0619. Min Act: -0.1700

Data Index: 7596917 (Open Web Text)

Max Activating Token Index: 665

Text #4

Max Range: 3.1392. Min Range: -3.1392

Max Act: 3.0613. Min Act: -0.1700

Data Index: 7433574 (Open Web Text)

Max Activating Token Index: 116

Text #5

Max Range: 3.1392. Min Range: -3.1392

Max Act: 3.0368. Min Act: -0.1700

Data Index: 6566609 (Open Web Text)

Max Activating Token Index: 503

Text #6

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9773. Min Act: -0.1700

Data Index: 22150 (Open Web Text)

Max Activating Token Index: 60

Text #7

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9647. Min Act: -0.1700

Data Index: 5340771 (Open Web Text)

Max Activating Token Index: 488

Text #8

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9389. Min Act: -0.1700

Data Index: 5811711 (Open Web Text)

Max Activating Token Index: 611

Text #9

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9317. Min Act: -0.1700

Data Index: 2593185 (Open Web Text)

Max Activating Token Index: 727

Text #10

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9289. Min Act: -0.1700

Data Index: 634689 (Open Web Text)

Max Activating Token Index: 378

Text #11

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9132. Min Act: -0.1700

Data Index: 4506534 (Open Web Text)

Max Activating Token Index: 958

Text #12

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9166. Min Act: -0.1700

Data Index: 2542384 (Open Web Text)

Max Activating Token Index: 650

Text #13

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9111. Min Act: -0.1700

Data Index: 5500248 (Open Web Text)

Max Activating Token Index: 327

Text #14

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9137. Min Act: -0.1700

Data Index: 2980307 (Open Web Text)

Max Activating Token Index: 846

Text #15

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9331. Min Act: -0.1700

Data Index: 7827260 (Open Web Text)

Max Activating Token Index: 15

Text #16

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9059. Min Act: -0.1700

Data Index: 8507252 (Open Web Text)

Max Activating Token Index: 481

Text #17

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9197. Min Act: -0.1700

Data Index: 2813800 (Open Web Text)

Max Activating Token Index: 503

Text #18

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9021. Min Act: -0.1700

Data Index: 2434605 (Open Web Text)

Max Activating Token Index: 252

Text #19

Max Range: 3.1392. Min Range: -3.1392

Max Act: 2.9095. Min Act: -0.1700

Data Index: 3986692 (Open Web Text)

Max Activating Token Index: 127