Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2307 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.3104. Min Act: -0.1700

Data Index: 176312 (Open Web Text)

Max Activating Token Index: 7

Text #1

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1998. Min Act: -0.1700

Data Index: 1167968 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1546. Min Act: -0.1700

Data Index: 2361340 (Open Web Text)

Max Activating Token Index: 58

Text #3

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1827. Min Act: -0.1700

Data Index: 2030443 (Open Web Text)

Max Activating Token Index: 308

Text #4

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1834. Min Act: -0.1700

Data Index: 6968958 (Open Web Text)

Max Activating Token Index: 277

Text #5

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1691. Min Act: -0.1700

Data Index: 2629180 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1262. Min Act: -0.1700

Data Index: 8681338 (Open Web Text)

Max Activating Token Index: 97

Text #7

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1189. Min Act: -0.1700

Data Index: 104454 (Open Web Text)

Max Activating Token Index: 378

Text #8

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1037. Min Act: -0.1700

Data Index: 3220543 (Open Web Text)

Max Activating Token Index: 137

Text #9

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1078. Min Act: -0.1700

Data Index: 7797771 (Open Web Text)

Max Activating Token Index: 226

Text #10

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.0983. Min Act: -0.1700

Data Index: 5013560 (Open Web Text)

Max Activating Token Index: 132

Text #11

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1141. Min Act: -0.1700

Data Index: 4766766 (Open Web Text)

Max Activating Token Index: 653

Text #12

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1450. Min Act: -0.1700

Data Index: 5731907 (Open Web Text)

Max Activating Token Index: 461

Text #13

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.0835. Min Act: -0.1700

Data Index: 7408128 (Open Web Text)

Max Activating Token Index: 154

Text #14

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.0835. Min Act: -0.1700

Data Index: 8245614 (Open Web Text)

Max Activating Token Index: 167

Text #15

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.0986. Min Act: -0.1700

Data Index: 5944583 (Open Web Text)

Max Activating Token Index: 7

Text #16

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.1011. Min Act: -0.1700

Data Index: 8342371 (Open Web Text)

Max Activating Token Index: 15

Text #17

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.0987. Min Act: -0.1700

Data Index: 4329534 (Open Web Text)

Max Activating Token Index: 238

Text #18

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.0772. Min Act: -0.1700

Data Index: 5380514 (Open Web Text)

Max Activating Token Index: 257

Text #19

Max Range: 4.3104. Min Range: -4.3104

Max Act: 4.0852. Min Act: -0.1700

Data Index: 5525301 (Open Web Text)

Max Activating Token Index: 9