Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 179 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1330. Min Range: -4.1330

Max Act: 4.1330. Min Act: -0.1700

Data Index: 7417170 (Open Web Text)

Max Activating Token Index: 414

Text #1

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.9794. Min Act: -0.1700

Data Index: 2648724 (Open Web Text)

Max Activating Token Index: 560

Text #2

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.9558. Min Act: -0.1700

Data Index: 7534991 (Open Web Text)

Max Activating Token Index: 165

Text #3

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.8009. Min Act: -0.1700

Data Index: 8515663 (Open Web Text)

Max Activating Token Index: 248

Text #4

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.7730. Min Act: -0.1700

Data Index: 8077703 (Open Web Text)

Max Activating Token Index: 628

Text #5

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.8401. Min Act: -0.1700

Data Index: 8235296 (Open Web Text)

Max Activating Token Index: 577

Text #6

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.7600. Min Act: -0.1700

Data Index: 8282336 (Open Web Text)

Max Activating Token Index: 483

Text #7

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6899. Min Act: -0.1700

Data Index: 1479743 (Open Web Text)

Max Activating Token Index: 177

Text #8

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6331. Min Act: -0.1700

Data Index: 2045864 (Open Web Text)

Max Activating Token Index: 399

Text #9

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6983. Min Act: -0.1700

Data Index: 2576617 (Open Web Text)

Max Activating Token Index: 635

Text #10

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6947. Min Act: -0.1700

Data Index: 2016868 (Open Web Text)

Max Activating Token Index: 514

Text #11

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6126. Min Act: -0.1700

Data Index: 4389977 (Open Web Text)

Max Activating Token Index: 335

Text #12

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6561. Min Act: -0.1699

Data Index: 6098985 (Open Web Text)

Max Activating Token Index: 168

Text #13

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6767. Min Act: -0.1700

Data Index: 402649 (Open Web Text)

Max Activating Token Index: 130

Text #14

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6288. Min Act: -0.1700

Data Index: 6360400 (Open Web Text)

Max Activating Token Index: 445

Text #15

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6065. Min Act: -0.1700

Data Index: 8565971 (Open Web Text)

Max Activating Token Index: 106

Text #16

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6504. Min Act: -0.1700

Data Index: 449537 (Open Web Text)

Max Activating Token Index: 927

Text #17

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.6190. Min Act: -0.1700

Data Index: 4117117 (Open Web Text)

Max Activating Token Index: 392

Text #18

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.5937. Min Act: -0.1700

Data Index: 8050440 (Open Web Text)

Max Activating Token Index: 100

Text #19

Max Range: 4.1330. Min Range: -4.1330

Max Act: 3.5789. Min Act: -0.1700

Data Index: 6093950 (Open Web Text)

Max Activating Token Index: 475