Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 94 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0599. Min Range: -4.0599

Max Act: 4.0599. Min Act: -0.1700

Data Index: 5940534 (Open Web Text)

Max Activating Token Index: 194

Text #1

Max Range: 4.0599. Min Range: -4.0599

Max Act: 4.0350. Min Act: -0.1700

Data Index: 4351368 (Open Web Text)

Max Activating Token Index: 348

Text #2

Max Range: 4.0599. Min Range: -4.0599

Max Act: 4.0247. Min Act: -0.1700

Data Index: 8794265 (Open Web Text)

Max Activating Token Index: 259

Text #3

Max Range: 4.0599. Min Range: -4.0599

Max Act: 4.0066. Min Act: -0.1700

Data Index: 4367281 (Open Web Text)

Max Activating Token Index: 290

Text #4

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9913. Min Act: -0.1700

Data Index: 8724024 (Open Web Text)

Max Activating Token Index: 227

Text #5

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9990. Min Act: -0.1700

Data Index: 739243 (Open Web Text)

Max Activating Token Index: 489

Text #6

Max Range: 4.0599. Min Range: -4.0599

Max Act: 4.0140. Min Act: -0.1700

Data Index: 6009437 (Open Web Text)

Max Activating Token Index: 450

Text #7

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9870. Min Act: -0.1700

Data Index: 6665631 (Open Web Text)

Max Activating Token Index: 598

Text #8

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9769. Min Act: -0.1700

Data Index: 7879105 (Open Web Text)

Max Activating Token Index: 488

Text #9

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9926. Min Act: -0.1700

Data Index: 931953 (Open Web Text)

Max Activating Token Index: 106

Text #10

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9517. Min Act: -0.1700

Data Index: 4830247 (Open Web Text)

Max Activating Token Index: 314

Text #11

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9503. Min Act: -0.1700

Data Index: 7555518 (Open Web Text)

Max Activating Token Index: 505

Text #12

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9392. Min Act: -0.1700

Data Index: 4824142 (Open Web Text)

Max Activating Token Index: 185

Text #13

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9370. Min Act: -0.1700

Data Index: 1241551 (Open Web Text)

Max Activating Token Index: 370

Text #14

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9617. Min Act: -0.1700

Data Index: 1857530 (Open Web Text)

Max Activating Token Index: 952

Text #15

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9019. Min Act: -0.1700

Data Index: 1456289 (Open Web Text)

Max Activating Token Index: 511

Text #16

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9159. Min Act: -0.1700

Data Index: 6147739 (Open Web Text)

Max Activating Token Index: 448

Text #17

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.8994. Min Act: -0.1700

Data Index: 4035145 (Open Web Text)

Max Activating Token Index: 851

Text #18

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9159. Min Act: -0.1700

Data Index: 301447 (Open Web Text)

Max Activating Token Index: 337

Text #19

Max Range: 4.0599. Min Range: -4.0599

Max Act: 3.9021. Min Act: -0.1700

Data Index: 8749022 (Open Web Text)

Max Activating Token Index: 118