Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 574 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.4010. Min Act: -0.1700

Data Index: 4847932 (Open Web Text)

Max Activating Token Index: 812

Text #1

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.3422. Min Act: -0.1700

Data Index: 4500874 (Open Web Text)

Max Activating Token Index: 321

Text #2

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.2399. Min Act: -0.1700

Data Index: 1038338 (Open Web Text)

Max Activating Token Index: 921

Text #3

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.2705. Min Act: -0.1700

Data Index: 8515378 (Open Web Text)

Max Activating Token Index: 589

Text #4

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.2176. Min Act: -0.1700

Data Index: 7198532 (Open Web Text)

Max Activating Token Index: 186

Text #5

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.2205. Min Act: -0.1700

Data Index: 7434694 (Open Web Text)

Max Activating Token Index: 214

Text #6

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1905. Min Act: -0.1700

Data Index: 2449157 (Open Web Text)

Max Activating Token Index: 81

Text #7

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1364. Min Act: -0.1700

Data Index: 6389990 (Open Web Text)

Max Activating Token Index: 305

Text #8

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.2001. Min Act: -0.1700

Data Index: 327959 (Open Web Text)

Max Activating Token Index: 215

Text #9

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1798. Min Act: -0.1700

Data Index: 8758417 (Open Web Text)

Max Activating Token Index: 348

Text #10

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.2192. Min Act: -0.1700

Data Index: 3563098 (Open Web Text)

Max Activating Token Index: 599

Text #11

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1931. Min Act: -0.1700

Data Index: 8044272 (Open Web Text)

Max Activating Token Index: 713

Text #12

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1802. Min Act: -0.1700

Data Index: 4232769 (Open Web Text)

Max Activating Token Index: 244

Text #13

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1740. Min Act: -0.1700

Data Index: 7975660 (Open Web Text)

Max Activating Token Index: 189

Text #14

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1571. Min Act: -0.1700

Data Index: 7048585 (Open Web Text)

Max Activating Token Index: 884

Text #15

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1472. Min Act: -0.1700

Data Index: 8728427 (Open Web Text)

Max Activating Token Index: 232

Text #16

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1504. Min Act: -0.1700

Data Index: 1557687 (Open Web Text)

Max Activating Token Index: 555

Text #17

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1674. Min Act: -0.1700

Data Index: 7290943 (Open Web Text)

Max Activating Token Index: 235

Text #18

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1617. Min Act: -0.1700

Data Index: 3186816 (Open Web Text)

Max Activating Token Index: 468

Text #19

Max Range: 3.4010. Min Range: -3.4010

Max Act: 3.1374. Min Act: -0.1700

Data Index: 2328441 (Open Web Text)

Max Activating Token Index: 766