Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 324 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.3561. Min Act: -0.1700

Data Index: 573489 (Open Web Text)

Max Activating Token Index: 834

Text #1

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.3396. Min Act: -0.1700

Data Index: 6964761 (Open Web Text)

Max Activating Token Index: 607

Text #2

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.2907. Min Act: -0.1700

Data Index: 3551900 (Open Web Text)

Max Activating Token Index: 889

Text #3

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.2387. Min Act: -0.1700

Data Index: 3081660 (Open Web Text)

Max Activating Token Index: 848

Text #4

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.2501. Min Act: -0.1700

Data Index: 4991742 (Open Web Text)

Max Activating Token Index: 818

Text #5

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.2254. Min Act: -0.1700

Data Index: 8182036 (Open Web Text)

Max Activating Token Index: 930

Text #6

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1957. Min Act: -0.1700

Data Index: 4211457 (Open Web Text)

Max Activating Token Index: 796

Text #7

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1887. Min Act: -0.1700

Data Index: 6646058 (Open Web Text)

Max Activating Token Index: 803

Text #8

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1937. Min Act: -0.1700

Data Index: 1519676 (Open Web Text)

Max Activating Token Index: 858

Text #9

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1778. Min Act: -0.1700

Data Index: 1601277 (Open Web Text)

Max Activating Token Index: 43

Text #10

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1858. Min Act: -0.1700

Data Index: 2872986 (Open Web Text)

Max Activating Token Index: 594

Text #11

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1673. Min Act: -0.1700

Data Index: 5505631 (Open Web Text)

Max Activating Token Index: 123

Text #12

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1712. Min Act: -0.1700

Data Index: 7462956 (Open Web Text)

Max Activating Token Index: 120

Text #13

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1726. Min Act: -0.1700

Data Index: 3235096 (Open Web Text)

Max Activating Token Index: 640

Text #14

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1718. Min Act: -0.1700

Data Index: 8450147 (Open Web Text)

Max Activating Token Index: 843

Text #15

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1593. Min Act: -0.1700

Data Index: 4816171 (Open Web Text)

Max Activating Token Index: 561

Text #16

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1647. Min Act: -0.1700

Data Index: 7027944 (Open Web Text)

Max Activating Token Index: 371

Text #17

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1688. Min Act: -0.1700

Data Index: 8603852 (Open Web Text)

Max Activating Token Index: 299

Text #18

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1560. Min Act: -0.1700

Data Index: 707161 (Open Web Text)

Max Activating Token Index: 732

Text #19

Max Range: 3.3561. Min Range: -3.3561

Max Act: 3.1382. Min Act: -0.1700

Data Index: 4698834 (Open Web Text)

Max Activating Token Index: 400