Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1789 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.9902. Min Act: -0.1700

Data Index: 1020886 (Open Web Text)

Max Activating Token Index: 622

Text #1

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.9990. Min Act: -0.1700

Data Index: 4299621 (Open Web Text)

Max Activating Token Index: 958

Text #2

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.9351. Min Act: -0.1700

Data Index: 5939624 (Open Web Text)

Max Activating Token Index: 407

Text #3

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.9331. Min Act: -0.1700

Data Index: 804481 (Open Web Text)

Max Activating Token Index: 134

Text #4

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.9187. Min Act: -0.1700

Data Index: 7702655 (Open Web Text)

Max Activating Token Index: 199

Text #5

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.9114. Min Act: -0.1700

Data Index: 4393338 (Open Web Text)

Max Activating Token Index: 102

Text #6

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8904. Min Act: -0.1700

Data Index: 2693541 (Open Web Text)

Max Activating Token Index: 328

Text #7

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.9007. Min Act: -0.1700

Data Index: 4202296 (Open Web Text)

Max Activating Token Index: 789

Text #8

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8963. Min Act: -0.1700

Data Index: 1016002 (Open Web Text)

Max Activating Token Index: 738

Text #9

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8872. Min Act: -0.1700

Data Index: 3122485 (Open Web Text)

Max Activating Token Index: 139

Text #10

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8813. Min Act: -0.1700

Data Index: 6261837 (Open Web Text)

Max Activating Token Index: 684

Text #11

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8808. Min Act: -0.1700

Data Index: 5420950 (Open Web Text)

Max Activating Token Index: 33

Text #12

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8757. Min Act: -0.1700

Data Index: 2103547 (Open Web Text)

Max Activating Token Index: 28

Text #13

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8746. Min Act: -0.1700

Data Index: 1291901 (Open Web Text)

Max Activating Token Index: 11

Text #14

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8733. Min Act: -0.1700

Data Index: 6147743 (Open Web Text)

Max Activating Token Index: 518

Text #15

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8670. Min Act: -0.1700

Data Index: 846978 (Open Web Text)

Max Activating Token Index: 429

Text #16

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8592. Min Act: -0.1700

Data Index: 5888267 (Open Web Text)

Max Activating Token Index: 477

Text #17

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8618. Min Act: -0.1700

Data Index: 1334219 (Open Web Text)

Max Activating Token Index: 14

Text #18

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8625. Min Act: -0.1700

Data Index: 1612825 (Open Web Text)

Max Activating Token Index: 666

Text #19

Max Range: 1.9990. Min Range: -1.9990

Max Act: 1.8607. Min Act: -0.1700

Data Index: 5974208 (Open Web Text)

Max Activating Token Index: 666