Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1969 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.8558. Min Act: -0.1699

Data Index: 1998157 (Open Web Text)

Max Activating Token Index: 39

Text #1

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.8402. Min Act: -0.1697

Data Index: 5801519 (Open Web Text)

Max Activating Token Index: 25

Text #2

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.7581. Min Act: -0.1699

Data Index: 5071270 (Open Web Text)

Max Activating Token Index: 100

Text #3

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6763. Min Act: -0.1700

Data Index: 373450 (Open Web Text)

Max Activating Token Index: 39

Text #4

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6850. Min Act: -0.1700

Data Index: 3604963 (Open Web Text)

Max Activating Token Index: 106

Text #5

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6937. Min Act: -0.1696

Data Index: 889822 (Open Web Text)

Max Activating Token Index: 23

Text #6

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6990. Min Act: -0.1599

Data Index: 1495210 (Open Web Text)

Max Activating Token Index: 783

Text #7

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6907. Min Act: -0.1653

Data Index: 5704796 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6084. Min Act: -0.1661

Data Index: 1867569 (Open Web Text)

Max Activating Token Index: 50

Text #9

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6336. Min Act: -0.1700

Data Index: 5657930 (Open Web Text)

Max Activating Token Index: 85

Text #10

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6418. Min Act: -0.1700

Data Index: 2039528 (Open Web Text)

Max Activating Token Index: 42

Text #11

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6248. Min Act: -0.1700

Data Index: 6462442 (Open Web Text)

Max Activating Token Index: 17

Text #12

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6263. Min Act: -0.1700

Data Index: 1676706 (Open Web Text)

Max Activating Token Index: 63

Text #13

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6179. Min Act: -0.1700

Data Index: 3605307 (Open Web Text)

Max Activating Token Index: 52

Text #14

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6058. Min Act: -0.1700

Data Index: 6360727 (Open Web Text)

Max Activating Token Index: 627

Text #15

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.5953. Min Act: -0.0722

Data Index: 8765351 (Open Web Text)

Max Activating Token Index: 20

Text #16

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6003. Min Act: -0.1700

Data Index: 1867574 (Open Web Text)

Max Activating Token Index: 215

Text #17

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.5905. Min Act: -0.1700

Data Index: 5219340 (Open Web Text)

Max Activating Token Index: 35

Text #18

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.6114. Min Act: -0.1700

Data Index: 5434039 (Open Web Text)

Max Activating Token Index: 488

Text #19

Max Range: 5.8558. Min Range: -5.8558

Max Act: 5.5607. Min Act: -0.1693

Data Index: 2246029 (Open Web Text)

Max Activating Token Index: 72