Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 185 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.4846. Min Act: -0.1700

Data Index: 6063279 (Open Web Text)

Max Activating Token Index: 457

Text #1

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.4229. Min Act: -0.1700

Data Index: 960602 (Open Web Text)

Max Activating Token Index: 213

Text #2

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.2501. Min Act: -0.1700

Data Index: 7451496 (Open Web Text)

Max Activating Token Index: 135

Text #3

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.2065. Min Act: -0.1700

Data Index: 392701 (Open Web Text)

Max Activating Token Index: 246

Text #4

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.2458. Min Act: -0.1700

Data Index: 8715621 (Open Web Text)

Max Activating Token Index: 993

Text #5

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.2068. Min Act: -0.1700

Data Index: 5322766 (Open Web Text)

Max Activating Token Index: 342

Text #6

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.2158. Min Act: -0.1700

Data Index: 3346449 (Open Web Text)

Max Activating Token Index: 86

Text #7

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.1556. Min Act: -0.1700

Data Index: 5595166 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.1672. Min Act: -0.1700

Data Index: 8180490 (Open Web Text)

Max Activating Token Index: 145

Text #9

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.1665. Min Act: -0.1700

Data Index: 2352922 (Open Web Text)

Max Activating Token Index: 220

Text #10

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.1591. Min Act: -0.1700

Data Index: 4128721 (Open Web Text)

Max Activating Token Index: 88

Text #11

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.1861. Min Act: -0.1700

Data Index: 7266578 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.1446. Min Act: -0.1700

Data Index: 6960345 (Open Web Text)

Max Activating Token Index: 251

Text #13

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.0999. Min Act: -0.1700

Data Index: 7380575 (Open Web Text)

Max Activating Token Index: 306

Text #14

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.1000. Min Act: -0.1700

Data Index: 4614911 (Open Web Text)

Max Activating Token Index: 234

Text #15

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.0942. Min Act: -0.1700

Data Index: 786560 (Open Web Text)

Max Activating Token Index: 777

Text #16

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.1317. Min Act: -0.1700

Data Index: 7533907 (Open Web Text)

Max Activating Token Index: 1013

Text #17

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.0884. Min Act: -0.1700

Data Index: 8447297 (Open Web Text)

Max Activating Token Index: 409

Text #18

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.0949. Min Act: -0.1700

Data Index: 5572309 (Open Web Text)

Max Activating Token Index: 1000

Text #19

Max Range: 4.4846. Min Range: -4.4846

Max Act: 4.0782. Min Act: -0.1700

Data Index: 8572007 (Open Web Text)

Max Activating Token Index: 786