Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1077 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3954. Min Act: -0.1700

Data Index: 4413232 (Open Web Text)

Max Activating Token Index: 749

Text #1

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.4220. Min Act: -0.1700

Data Index: 2529294 (Open Web Text)

Max Activating Token Index: 607

Text #2

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3859. Min Act: -0.1700

Data Index: 3300307 (Open Web Text)

Max Activating Token Index: 554

Text #3

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3972. Min Act: -0.1700

Data Index: 588565 (Open Web Text)

Max Activating Token Index: 35

Text #4

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3625. Min Act: -0.1700

Data Index: 8231568 (Open Web Text)

Max Activating Token Index: 292

Text #5

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3776. Min Act: -0.1700

Data Index: 8595711 (Open Web Text)

Max Activating Token Index: 415

Text #6

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3584. Min Act: -0.1700

Data Index: 1590462 (Open Web Text)

Max Activating Token Index: 137

Text #7

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3761. Min Act: -0.1700

Data Index: 5755374 (Open Web Text)

Max Activating Token Index: 634

Text #8

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3748. Min Act: -0.1700

Data Index: 4124265 (Open Web Text)

Max Activating Token Index: 432

Text #9

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3290. Min Act: -0.1700

Data Index: 4989816 (Open Web Text)

Max Activating Token Index: 706

Text #10

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3534. Min Act: -0.1700

Data Index: 4240907 (Open Web Text)

Max Activating Token Index: 488

Text #11

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3547. Min Act: -0.1700

Data Index: 6514264 (Open Web Text)

Max Activating Token Index: 624

Text #12

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3503. Min Act: -0.1700

Data Index: 6590197 (Open Web Text)

Max Activating Token Index: 796

Text #13

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3322. Min Act: -0.1700

Data Index: 433676 (Open Web Text)

Max Activating Token Index: 424

Text #14

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3261. Min Act: -0.1700

Data Index: 603497 (Open Web Text)

Max Activating Token Index: 40

Text #15

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3321. Min Act: -0.1700

Data Index: 384567 (Open Web Text)

Max Activating Token Index: 153

Text #16

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3433. Min Act: -0.1700

Data Index: 846352 (Open Web Text)

Max Activating Token Index: 271

Text #17

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3096. Min Act: -0.1700

Data Index: 5290644 (Open Web Text)

Max Activating Token Index: 620

Text #18

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3130. Min Act: -0.1700

Data Index: 2800092 (Open Web Text)

Max Activating Token Index: 585

Text #19

Max Range: 3.4220. Min Range: -3.4220

Max Act: 3.3156. Min Act: -0.1700

Data Index: 2701745 (Open Web Text)

Max Activating Token Index: 5