Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2181 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.4182. Min Act: -0.1700

Data Index: 2938996 (Open Web Text)

Max Activating Token Index: 68

Text #1

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.3165. Min Act: -0.1700

Data Index: 1403656 (Open Web Text)

Max Activating Token Index: 654

Text #2

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.2889. Min Act: -0.1700

Data Index: 3376460 (Open Web Text)

Max Activating Token Index: 22

Text #3

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.2795. Min Act: -0.1700

Data Index: 8405714 (Open Web Text)

Max Activating Token Index: 446

Text #4

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.2020. Min Act: -0.1700

Data Index: 2113713 (Open Web Text)

Max Activating Token Index: 450

Text #5

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1821. Min Act: -0.1700

Data Index: 1681781 (Open Web Text)

Max Activating Token Index: 614

Text #6

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1586. Min Act: -0.1700

Data Index: 7701714 (Open Web Text)

Max Activating Token Index: 113

Text #7

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1630. Min Act: -0.1700

Data Index: 8792070 (Open Web Text)

Max Activating Token Index: 738

Text #8

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1847. Min Act: -0.1700

Data Index: 6040148 (Open Web Text)

Max Activating Token Index: 55

Text #9

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1475. Min Act: -0.1700

Data Index: 2984553 (Open Web Text)

Max Activating Token Index: 352

Text #10

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1282. Min Act: -0.1700

Data Index: 8757743 (Open Web Text)

Max Activating Token Index: 27

Text #11

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1496. Min Act: -0.1700

Data Index: 1353715 (Open Web Text)

Max Activating Token Index: 899

Text #12

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1246. Min Act: -0.1700

Data Index: 3463134 (Open Web Text)

Max Activating Token Index: 572

Text #13

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1322. Min Act: -0.1700

Data Index: 3765959 (Open Web Text)

Max Activating Token Index: 308

Text #14

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1213. Min Act: -0.1700

Data Index: 4379612 (Open Web Text)

Max Activating Token Index: 248

Text #15

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1261. Min Act: -0.1700

Data Index: 7200272 (Open Web Text)

Max Activating Token Index: 437

Text #16

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1231. Min Act: -0.1700

Data Index: 2943076 (Open Web Text)

Max Activating Token Index: 33

Text #17

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1176. Min Act: -0.1700

Data Index: 324825 (Open Web Text)

Max Activating Token Index: 234

Text #18

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.0847. Min Act: -0.1700

Data Index: 6399218 (Open Web Text)

Max Activating Token Index: 33

Text #19

Max Range: 4.4182. Min Range: -4.4182

Max Act: 4.1113. Min Act: -0.1700

Data Index: 6590448 (Open Web Text)

Max Activating Token Index: 294