Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2085 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.6597. Min Act: -0.1700

Data Index: 4357014 (Open Web Text)

Max Activating Token Index: 908

Text #1

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.6854. Min Act: -0.1700

Data Index: 3592773 (Open Web Text)

Max Activating Token Index: 647

Text #2

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.6648. Min Act: -0.1700

Data Index: 2773635 (Open Web Text)

Max Activating Token Index: 170

Text #3

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.6239. Min Act: -0.1700

Data Index: 7244007 (Open Web Text)

Max Activating Token Index: 512

Text #4

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5830. Min Act: -0.1700

Data Index: 2469064 (Open Web Text)

Max Activating Token Index: 858

Text #5

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5769. Min Act: -0.1700

Data Index: 3363449 (Open Web Text)

Max Activating Token Index: 271

Text #6

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5499. Min Act: -0.1700

Data Index: 4420265 (Open Web Text)

Max Activating Token Index: 498

Text #7

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5827. Min Act: -0.1700

Data Index: 3142155 (Open Web Text)

Max Activating Token Index: 416

Text #8

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5401. Min Act: -0.1700

Data Index: 3987017 (Open Web Text)

Max Activating Token Index: 328

Text #9

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5238. Min Act: -0.1700

Data Index: 5525027 (Open Web Text)

Max Activating Token Index: 875

Text #10

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5189. Min Act: -0.1700

Data Index: 3649933 (Open Web Text)

Max Activating Token Index: 333

Text #11

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.4618. Min Act: -0.1700

Data Index: 8502448 (Open Web Text)

Max Activating Token Index: 885

Text #12

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5262. Min Act: -0.1700

Data Index: 5077834 (Open Web Text)

Max Activating Token Index: 962

Text #13

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5088. Min Act: -0.1700

Data Index: 1253261 (Open Web Text)

Max Activating Token Index: 355

Text #14

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5186. Min Act: -0.1700

Data Index: 2624323 (Open Web Text)

Max Activating Token Index: 98

Text #15

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.5202. Min Act: -0.1700

Data Index: 5645686 (Open Web Text)

Max Activating Token Index: 983

Text #16

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.4637. Min Act: -0.1700

Data Index: 8531450 (Open Web Text)

Max Activating Token Index: 147

Text #17

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.4706. Min Act: -0.1700

Data Index: 1880163 (Open Web Text)

Max Activating Token Index: 106

Text #18

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.4585. Min Act: -0.1700

Data Index: 2313597 (Open Web Text)

Max Activating Token Index: 691

Text #19

Max Range: 4.6854. Min Range: -4.6854

Max Act: 4.4474. Min Act: -0.1700

Data Index: 2593656 (Open Web Text)

Max Activating Token Index: 837