Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 150 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.4444. Min Act: -0.1700

Data Index: 2333425 (Open Web Text)

Max Activating Token Index: 561

Text #1

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.4099. Min Act: -0.1700

Data Index: 3425570 (Open Web Text)

Max Activating Token Index: 546

Text #2

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.3354. Min Act: -0.1700

Data Index: 3645516 (Open Web Text)

Max Activating Token Index: 466

Text #3

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.2625. Min Act: -0.1700

Data Index: 3191651 (Open Web Text)

Max Activating Token Index: 305

Text #4

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.2760. Min Act: -0.1700

Data Index: 3413438 (Open Web Text)

Max Activating Token Index: 917

Text #5

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.1653. Min Act: -0.1700

Data Index: 4268424 (Open Web Text)

Max Activating Token Index: 995

Text #6

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.0815. Min Act: -0.1700

Data Index: 7552990 (Open Web Text)

Max Activating Token Index: 927

Text #7

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.1857. Min Act: -0.1700

Data Index: 3981233 (Open Web Text)

Max Activating Token Index: 818

Text #8

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.1148. Min Act: -0.1700

Data Index: 8348608 (Open Web Text)

Max Activating Token Index: 588

Text #9

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.0665. Min Act: -0.1700

Data Index: 887176 (Open Web Text)

Max Activating Token Index: 465

Text #10

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.0894. Min Act: -0.1700

Data Index: 2227802 (Open Web Text)

Max Activating Token Index: 636

Text #11

Max Range: 7.4444. Min Range: -7.4444

Max Act: 6.9975. Min Act: -0.1700

Data Index: 4223343 (Open Web Text)

Max Activating Token Index: 1014

Text #12

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.0112. Min Act: -0.1700

Data Index: 1239468 (Open Web Text)

Max Activating Token Index: 584

Text #13

Max Range: 7.4444. Min Range: -7.4444

Max Act: 6.9939. Min Act: -0.1700

Data Index: 4049347 (Open Web Text)

Max Activating Token Index: 231

Text #14

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.0485. Min Act: -0.1700

Data Index: 6663272 (Open Web Text)

Max Activating Token Index: 889

Text #15

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.0036. Min Act: -0.1700

Data Index: 2406928 (Open Web Text)

Max Activating Token Index: 636

Text #16

Max Range: 7.4444. Min Range: -7.4444

Max Act: 6.9882. Min Act: -0.1700

Data Index: 2413869 (Open Web Text)

Max Activating Token Index: 645

Text #17

Max Range: 7.4444. Min Range: -7.4444

Max Act: 6.9792. Min Act: -0.1700

Data Index: 391386 (Open Web Text)

Max Activating Token Index: 992

Text #18

Max Range: 7.4444. Min Range: -7.4444

Max Act: 7.0744. Min Act: -0.1700

Data Index: 2774806 (Open Web Text)

Max Activating Token Index: 675

Text #19

Max Range: 7.4444. Min Range: -7.4444

Max Act: 6.9612. Min Act: -0.1700

Data Index: 5690460 (Open Web Text)

Max Activating Token Index: 471