Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2718 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3834. Min Act: -0.1700

Data Index: 1253924 (Open Web Text)

Max Activating Token Index: 2

Text #1

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3847. Min Act: -0.1700

Data Index: 6631996 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3648. Min Act: -0.1700

Data Index: 6213742 (Open Web Text)

Max Activating Token Index: 2

Text #3

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3691. Min Act: -0.1700

Data Index: 2592642 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3426. Min Act: -0.1700

Data Index: 7455039 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3544. Min Act: -0.1700

Data Index: 8413696 (Open Web Text)

Max Activating Token Index: 9

Text #6

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3373. Min Act: -0.1700

Data Index: 8423482 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3519. Min Act: -0.1700

Data Index: 1345572 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3325. Min Act: -0.1700

Data Index: 3755010 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3260. Min Act: -0.1700

Data Index: 317603 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3273. Min Act: -0.1700

Data Index: 1306027 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3222. Min Act: -0.1697

Data Index: 1122012 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3333. Min Act: -0.1700

Data Index: 6308929 (Open Web Text)

Max Activating Token Index: 31

Text #13

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3240. Min Act: -0.1700

Data Index: 8413700 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3127. Min Act: -0.1700

Data Index: 1122013 (Open Web Text)

Max Activating Token Index: 8

Text #15

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3179. Min Act: -0.1700

Data Index: 6170712 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3187. Min Act: -0.1700

Data Index: 6468274 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3107. Min Act: -0.1700

Data Index: 1151436 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3109. Min Act: -0.1700

Data Index: 5891589 (Open Web Text)

Max Activating Token Index: 307

Text #19

Max Range: 2.3847. Min Range: -2.3847

Max Act: 2.3190. Min Act: -0.1700

Data Index: 6883329 (Open Web Text)

Max Activating Token Index: 6