Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1036 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.9093. Min Act: -0.1700

Data Index: 2548970 (Open Web Text)

Max Activating Token Index: 895

Text #1

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.8370. Min Act: -0.1700

Data Index: 3810459 (Open Web Text)

Max Activating Token Index: 876

Text #2

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.8612. Min Act: -0.1700

Data Index: 5854168 (Open Web Text)

Max Activating Token Index: 841

Text #3

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.8219. Min Act: -0.1700

Data Index: 7371062 (Open Web Text)

Max Activating Token Index: 111

Text #4

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.8159. Min Act: -0.1700

Data Index: 7855706 (Open Web Text)

Max Activating Token Index: 961

Text #5

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7671. Min Act: -0.1700

Data Index: 362410 (Open Web Text)

Max Activating Token Index: 44

Text #6

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.8170. Min Act: -0.1700

Data Index: 6879493 (Open Web Text)

Max Activating Token Index: 607

Text #7

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.8105. Min Act: -0.1700

Data Index: 1796852 (Open Web Text)

Max Activating Token Index: 673

Text #8

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.8221. Min Act: -0.1700

Data Index: 8717894 (Open Web Text)

Max Activating Token Index: 7

Text #9

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7793. Min Act: -0.1700

Data Index: 6619918 (Open Web Text)

Max Activating Token Index: 831

Text #10

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7855. Min Act: -0.1700

Data Index: 7904008 (Open Web Text)

Max Activating Token Index: 899

Text #11

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7549. Min Act: -0.1700

Data Index: 4797862 (Open Web Text)

Max Activating Token Index: 8

Text #12

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7588. Min Act: -0.1700

Data Index: 5598860 (Open Web Text)

Max Activating Token Index: 913

Text #13

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7683. Min Act: -0.1700

Data Index: 4154764 (Open Web Text)

Max Activating Token Index: 548

Text #14

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7450. Min Act: -0.1700

Data Index: 3147045 (Open Web Text)

Max Activating Token Index: 148

Text #15

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7350. Min Act: -0.1700

Data Index: 3330886 (Open Web Text)

Max Activating Token Index: 866

Text #16

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7480. Min Act: -0.1700

Data Index: 5116255 (Open Web Text)

Max Activating Token Index: 711

Text #17

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7490. Min Act: -0.1700

Data Index: 4282951 (Open Web Text)

Max Activating Token Index: 978

Text #18

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7112. Min Act: -0.1700

Data Index: 6211038 (Open Web Text)

Max Activating Token Index: 879

Text #19

Max Range: 4.9093. Min Range: -4.9093

Max Act: 4.7554. Min Act: -0.1700

Data Index: 563909 (Open Web Text)

Max Activating Token Index: 940