Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 761 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.2167. Min Act: -0.1700

Data Index: 7605824 (Open Web Text)

Max Activating Token Index: 443

Text #1

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.2137. Min Act: -0.1700

Data Index: 2145379 (Open Web Text)

Max Activating Token Index: 257

Text #2

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1871. Min Act: -0.1700

Data Index: 3424193 (Open Web Text)

Max Activating Token Index: 113

Text #3

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1930. Min Act: -0.1700

Data Index: 1580940 (Open Web Text)

Max Activating Token Index: 299

Text #4

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.2013. Min Act: -0.1700

Data Index: 5414477 (Open Web Text)

Max Activating Token Index: 144

Text #5

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1874. Min Act: -0.1700

Data Index: 5828302 (Open Web Text)

Max Activating Token Index: 240

Text #6

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1921. Min Act: -0.1700

Data Index: 1669530 (Open Web Text)

Max Activating Token Index: 138

Text #7

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1992. Min Act: -0.1700

Data Index: 4736943 (Open Web Text)

Max Activating Token Index: 138

Text #8

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.2199. Min Act: -0.1700

Data Index: 6747971 (Open Web Text)

Max Activating Token Index: 766

Text #9

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1738. Min Act: -0.1700

Data Index: 5877036 (Open Web Text)

Max Activating Token Index: 194

Text #10

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1663. Min Act: -0.1700

Data Index: 8116107 (Open Web Text)

Max Activating Token Index: 291

Text #11

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1520. Min Act: -0.1700

Data Index: 3192189 (Open Web Text)

Max Activating Token Index: 181

Text #12

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1621. Min Act: -0.1700

Data Index: 2883111 (Open Web Text)

Max Activating Token Index: 121

Text #13

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1739. Min Act: -0.1700

Data Index: 1771502 (Open Web Text)

Max Activating Token Index: 300

Text #14

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1695. Min Act: -0.1700

Data Index: 1642638 (Open Web Text)

Max Activating Token Index: 486

Text #15

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1735. Min Act: -0.1700

Data Index: 4868503 (Open Web Text)

Max Activating Token Index: 775

Text #16

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1602. Min Act: -0.1700

Data Index: 2712101 (Open Web Text)

Max Activating Token Index: 260

Text #17

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1855. Min Act: -0.1700

Data Index: 572474 (Open Web Text)

Max Activating Token Index: 28

Text #18

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1481. Min Act: -0.1700

Data Index: 3848318 (Open Web Text)

Max Activating Token Index: 176

Text #19

Max Range: 4.2199. Min Range: -4.2199

Max Act: 4.1459. Min Act: -0.1700

Data Index: 4090785 (Open Web Text)

Max Activating Token Index: 111