Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1306 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.7767. Min Act: -0.1700

Data Index: 1458540 (Open Web Text)

Max Activating Token Index: 129

Text #1

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.7499. Min Act: -0.1700

Data Index: 7951637 (Open Web Text)

Max Activating Token Index: 413

Text #2

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.6954. Min Act: -0.1700

Data Index: 8193863 (Open Web Text)

Max Activating Token Index: 199

Text #3

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5619. Min Act: -0.1700

Data Index: 4010875 (Open Web Text)

Max Activating Token Index: 477

Text #4

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5781. Min Act: -0.1700

Data Index: 5176039 (Open Web Text)

Max Activating Token Index: 776

Text #5

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5996. Min Act: -0.1700

Data Index: 147214 (Open Web Text)

Max Activating Token Index: 559

Text #6

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5844. Min Act: -0.1700

Data Index: 7916133 (Open Web Text)

Max Activating Token Index: 110

Text #7

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5463. Min Act: -0.1700

Data Index: 4003893 (Open Web Text)

Max Activating Token Index: 58

Text #8

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5186. Min Act: -0.1700

Data Index: 1149216 (Open Web Text)

Max Activating Token Index: 857

Text #9

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5597. Min Act: -0.1700

Data Index: 611899 (Open Web Text)

Max Activating Token Index: 883

Text #10

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5071. Min Act: -0.1700

Data Index: 4147551 (Open Web Text)

Max Activating Token Index: 269

Text #11

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.4903. Min Act: -0.1700

Data Index: 7557752 (Open Web Text)

Max Activating Token Index: 889

Text #12

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5200. Min Act: -0.1700

Data Index: 3199491 (Open Web Text)

Max Activating Token Index: 166

Text #13

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.4541. Min Act: -0.1700

Data Index: 3008223 (Open Web Text)

Max Activating Token Index: 679

Text #14

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5000. Min Act: -0.1700

Data Index: 2350546 (Open Web Text)

Max Activating Token Index: 386

Text #15

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.4946. Min Act: -0.1700

Data Index: 8387727 (Open Web Text)

Max Activating Token Index: 651

Text #16

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5052. Min Act: -0.1700

Data Index: 8138587 (Open Web Text)

Max Activating Token Index: 301

Text #17

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.5253. Min Act: -0.1700

Data Index: 3393537 (Open Web Text)

Max Activating Token Index: 887

Text #18

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.4565. Min Act: -0.1700

Data Index: 7265971 (Open Web Text)

Max Activating Token Index: 212

Text #19

Max Range: 6.7767. Min Range: -6.7767

Max Act: 6.4617. Min Act: -0.1700

Data Index: 1489445 (Open Web Text)

Max Activating Token Index: 136