Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1516 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.4950. Min Act: -0.1700

Data Index: 7990104 (Open Web Text)

Max Activating Token Index: 68

Text #1

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.3066. Min Act: -0.1700

Data Index: 3573616 (Open Web Text)

Max Activating Token Index: 389

Text #2

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.2477. Min Act: -0.1700

Data Index: 5897990 (Open Web Text)

Max Activating Token Index: 76

Text #3

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.2700. Min Act: -0.1700

Data Index: 2785440 (Open Web Text)

Max Activating Token Index: 19

Text #4

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.2646. Min Act: -0.1700

Data Index: 5603324 (Open Web Text)

Max Activating Token Index: 21

Text #5

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.2142. Min Act: -0.1700

Data Index: 883110 (Open Web Text)

Max Activating Token Index: 253

Text #6

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.2246. Min Act: -0.1700

Data Index: 4875808 (Open Web Text)

Max Activating Token Index: 54

Text #7

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.2254. Min Act: -0.1700

Data Index: 1572177 (Open Web Text)

Max Activating Token Index: 263

Text #8

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.2136. Min Act: -0.1700

Data Index: 7126529 (Open Web Text)

Max Activating Token Index: 404

Text #9

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.2280. Min Act: -0.1700

Data Index: 2367057 (Open Web Text)

Max Activating Token Index: 29

Text #10

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.1793. Min Act: -0.1700

Data Index: 4822273 (Open Web Text)

Max Activating Token Index: 259

Text #11

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.1934. Min Act: -0.1700

Data Index: 5100885 (Open Web Text)

Max Activating Token Index: 963

Text #12

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.2032. Min Act: -0.1700

Data Index: 6246248 (Open Web Text)

Max Activating Token Index: 70

Text #13

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.1935. Min Act: -0.1700

Data Index: 1412664 (Open Web Text)

Max Activating Token Index: 342

Text #14

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.1999. Min Act: -0.1700

Data Index: 1275889 (Open Web Text)

Max Activating Token Index: 418

Text #15

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.1974. Min Act: -0.1700

Data Index: 3205375 (Open Web Text)

Max Activating Token Index: 19

Text #16

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.1996. Min Act: -0.1700

Data Index: 5773370 (Open Web Text)

Max Activating Token Index: 120

Text #17

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.1711. Min Act: -0.1700

Data Index: 814755 (Open Web Text)

Max Activating Token Index: 977

Text #18

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.1548. Min Act: -0.1700

Data Index: 5279645 (Open Web Text)

Max Activating Token Index: 887

Text #19

Max Range: 4.4950. Min Range: -4.4950

Max Act: 4.1647. Min Act: -0.1700

Data Index: 1153174 (Open Web Text)

Max Activating Token Index: 502