Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 408 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.5077. Min Act: -0.1700

Data Index: 4153667 (Open Web Text)

Max Activating Token Index: 524

Text #1

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.4765. Min Act: -0.1700

Data Index: 6250631 (Open Web Text)

Max Activating Token Index: 254

Text #2

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.4264. Min Act: -0.1700

Data Index: 2290642 (Open Web Text)

Max Activating Token Index: 354

Text #3

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.4283. Min Act: -0.1700

Data Index: 3455923 (Open Web Text)

Max Activating Token Index: 497

Text #4

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.4410. Min Act: -0.1700

Data Index: 6107394 (Open Web Text)

Max Activating Token Index: 751

Text #5

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3908. Min Act: -0.1700

Data Index: 855919 (Open Web Text)

Max Activating Token Index: 633

Text #6

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3727. Min Act: -0.1700

Data Index: 3956253 (Open Web Text)

Max Activating Token Index: 515

Text #7

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3451. Min Act: -0.1700

Data Index: 6682609 (Open Web Text)

Max Activating Token Index: 774

Text #8

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3308. Min Act: -0.1700

Data Index: 4651365 (Open Web Text)

Max Activating Token Index: 633

Text #9

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3103. Min Act: -0.1700

Data Index: 5591073 (Open Web Text)

Max Activating Token Index: 564

Text #10

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3383. Min Act: -0.1700

Data Index: 5589166 (Open Web Text)

Max Activating Token Index: 331

Text #11

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3241. Min Act: -0.1700

Data Index: 2437205 (Open Web Text)

Max Activating Token Index: 385

Text #12

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3225. Min Act: -0.1700

Data Index: 977300 (Open Web Text)

Max Activating Token Index: 243

Text #13

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.2973. Min Act: -0.1700

Data Index: 6478926 (Open Web Text)

Max Activating Token Index: 639

Text #14

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3030. Min Act: -0.1700

Data Index: 3191081 (Open Web Text)

Max Activating Token Index: 224

Text #15

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.2941. Min Act: -0.1700

Data Index: 7553939 (Open Web Text)

Max Activating Token Index: 379

Text #16

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3011. Min Act: -0.1700

Data Index: 5406749 (Open Web Text)

Max Activating Token Index: 527

Text #17

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3098. Min Act: -0.1700

Data Index: 4252609 (Open Web Text)

Max Activating Token Index: 566

Text #18

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.3142. Min Act: -0.1700

Data Index: 5615285 (Open Web Text)

Max Activating Token Index: 390

Text #19

Max Range: 3.5077. Min Range: -3.5077

Max Act: 3.2945. Min Act: -0.1700

Data Index: 6006171 (Open Web Text)

Max Activating Token Index: 212