Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2215 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.6716. Min Act: -0.1700

Data Index: 5437769 (Open Web Text)

Max Activating Token Index: 326

Text #1

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.2167. Min Act: -0.1700

Data Index: 1539270 (Open Web Text)

Max Activating Token Index: 532

Text #2

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.2092. Min Act: -0.1700

Data Index: 544665 (Open Web Text)

Max Activating Token Index: 55

Text #3

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.1562. Min Act: -0.1700

Data Index: 3859831 (Open Web Text)

Max Activating Token Index: 478

Text #4

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.1544. Min Act: -0.1700

Data Index: 5543520 (Open Web Text)

Max Activating Token Index: 860

Text #5

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.1480. Min Act: -0.1700

Data Index: 914854 (Open Web Text)

Max Activating Token Index: 248

Text #6

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.1133. Min Act: -0.1700

Data Index: 8817690 (Open Web Text)

Max Activating Token Index: 27

Text #7

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.0954. Min Act: -0.1700

Data Index: 5063684 (Open Web Text)

Max Activating Token Index: 166

Text #8

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.0552. Min Act: -0.1700

Data Index: 2625795 (Open Web Text)

Max Activating Token Index: 69

Text #9

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.0555. Min Act: -0.1700

Data Index: 7313455 (Open Web Text)

Max Activating Token Index: 37

Text #10

Max Range: 5.6716. Min Range: -5.6716

Max Act: 4.9863. Min Act: -0.1700

Data Index: 5789069 (Open Web Text)

Max Activating Token Index: 64

Text #11

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.0056. Min Act: -0.1700

Data Index: 8795447 (Open Web Text)

Max Activating Token Index: 67

Text #12

Max Range: 5.6716. Min Range: -5.6716

Max Act: 4.9945. Min Act: -0.1700

Data Index: 8817687 (Open Web Text)

Max Activating Token Index: 94

Text #13

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.0020. Min Act: -0.1700

Data Index: 4690120 (Open Web Text)

Max Activating Token Index: 21

Text #14

Max Range: 5.6716. Min Range: -5.6716

Max Act: 5.0217. Min Act: -0.1700

Data Index: 7861977 (Open Web Text)

Max Activating Token Index: 63

Text #15

Max Range: 5.6716. Min Range: -5.6716

Max Act: 4.9707. Min Act: -0.1700

Data Index: 2473035 (Open Web Text)

Max Activating Token Index: 69

Text #16

Max Range: 5.6716. Min Range: -5.6716

Max Act: 4.9815. Min Act: -0.1700

Data Index: 6517634 (Open Web Text)

Max Activating Token Index: 449

Text #17

Max Range: 5.6716. Min Range: -5.6716

Max Act: 4.9999. Min Act: -0.1700

Data Index: 185987 (Open Web Text)

Max Activating Token Index: 25

Text #18

Max Range: 5.6716. Min Range: -5.6716

Max Act: 4.9730. Min Act: -0.1700

Data Index: 2073018 (Open Web Text)

Max Activating Token Index: 301

Text #19

Max Range: 5.6716. Min Range: -5.6716

Max Act: 4.9553. Min Act: -0.1700

Data Index: 6135380 (Open Web Text)

Max Activating Token Index: 684