Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 390 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.4134. Min Act: -0.1700

Data Index: 5987335 (Open Web Text)

Max Activating Token Index: 446

Text #1

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.4268. Min Act: -0.1700

Data Index: 7695273 (Open Web Text)

Max Activating Token Index: 439

Text #2

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.4162. Min Act: -0.1700

Data Index: 5786227 (Open Web Text)

Max Activating Token Index: 228

Text #3

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.4352. Min Act: -0.1700

Data Index: 7211209 (Open Web Text)

Max Activating Token Index: 481

Text #4

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3689. Min Act: -0.1700

Data Index: 4248026 (Open Web Text)

Max Activating Token Index: 533

Text #5

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3978. Min Act: -0.1700

Data Index: 2365300 (Open Web Text)

Max Activating Token Index: 833

Text #6

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3938. Min Act: -0.1700

Data Index: 471428 (Open Web Text)

Max Activating Token Index: 408

Text #7

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3591. Min Act: -0.1700

Data Index: 6307812 (Open Web Text)

Max Activating Token Index: 390

Text #8

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3270. Min Act: -0.1700

Data Index: 7140548 (Open Web Text)

Max Activating Token Index: 27

Text #9

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3211. Min Act: -0.1700

Data Index: 1815705 (Open Web Text)

Max Activating Token Index: 393

Text #10

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3682. Min Act: -0.1700

Data Index: 1971319 (Open Web Text)

Max Activating Token Index: 973

Text #11

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3373. Min Act: -0.1700

Data Index: 3008793 (Open Web Text)

Max Activating Token Index: 608

Text #12

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3588. Min Act: -0.1700

Data Index: 7571906 (Open Web Text)

Max Activating Token Index: 660

Text #13

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3391. Min Act: -0.1700

Data Index: 6215984 (Open Web Text)

Max Activating Token Index: 876

Text #14

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3359. Min Act: -0.1700

Data Index: 1446752 (Open Web Text)

Max Activating Token Index: 411

Text #15

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3496. Min Act: -0.1700

Data Index: 3751630 (Open Web Text)

Max Activating Token Index: 998

Text #16

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3900. Min Act: -0.1700

Data Index: 2501259 (Open Web Text)

Max Activating Token Index: 605

Text #17

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3507. Min Act: -0.1700

Data Index: 2436242 (Open Web Text)

Max Activating Token Index: 795

Text #18

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.2926. Min Act: -0.1700

Data Index: 1487206 (Open Web Text)

Max Activating Token Index: 403

Text #19

Max Range: 4.4352. Min Range: -4.4352

Max Act: 4.3495. Min Act: -0.1700

Data Index: 4098032 (Open Web Text)

Max Activating Token Index: 608