Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1919 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.4186. Min Act: -0.1700

Data Index: 5944300 (Open Web Text)

Max Activating Token Index: 359

Text #1

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.4543. Min Act: -0.1700

Data Index: 7553001 (Open Web Text)

Max Activating Token Index: 371

Text #2

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.3917. Min Act: -0.1700

Data Index: 1741444 (Open Web Text)

Max Activating Token Index: 289

Text #3

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.3259. Min Act: -0.1700

Data Index: 5509329 (Open Web Text)

Max Activating Token Index: 582

Text #4

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.3462. Min Act: -0.1700

Data Index: 4551875 (Open Web Text)

Max Activating Token Index: 118

Text #5

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.3148. Min Act: -0.1700

Data Index: 6244856 (Open Web Text)

Max Activating Token Index: 419

Text #6

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.3167. Min Act: -0.1700

Data Index: 5251065 (Open Web Text)

Max Activating Token Index: 802

Text #7

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.3257. Min Act: -0.1700

Data Index: 7596872 (Open Web Text)

Max Activating Token Index: 475

Text #8

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.3255. Min Act: -0.1700

Data Index: 3947519 (Open Web Text)

Max Activating Token Index: 159

Text #9

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.3093. Min Act: -0.1700

Data Index: 6581205 (Open Web Text)

Max Activating Token Index: 781

Text #10

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.3042. Min Act: -0.1700

Data Index: 6881967 (Open Web Text)

Max Activating Token Index: 235

Text #11

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.3144. Min Act: -0.1700

Data Index: 3562825 (Open Web Text)

Max Activating Token Index: 869

Text #12

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.2636. Min Act: -0.1700

Data Index: 6762038 (Open Web Text)

Max Activating Token Index: 433

Text #13

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.2495. Min Act: -0.1700

Data Index: 876118 (Open Web Text)

Max Activating Token Index: 340

Text #14

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.2855. Min Act: -0.1700

Data Index: 3333348 (Open Web Text)

Max Activating Token Index: 658

Text #15

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.2729. Min Act: -0.1700

Data Index: 1230594 (Open Web Text)

Max Activating Token Index: 67

Text #16

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.2955. Min Act: -0.1700

Data Index: 759094 (Open Web Text)

Max Activating Token Index: 974

Text #17

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.2495. Min Act: -0.1700

Data Index: 1924400 (Open Web Text)

Max Activating Token Index: 64

Text #18

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.2371. Min Act: -0.1700

Data Index: 5613857 (Open Web Text)

Max Activating Token Index: 846

Text #19

Max Range: 4.4543. Min Range: -4.4543

Max Act: 4.2463. Min Act: -0.1700

Data Index: 576812 (Open Web Text)

Max Activating Token Index: 147