Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2964 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.2855. Min Act: -0.1700

Data Index: 4936070 (Open Web Text)

Max Activating Token Index: 54

Text #1

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.2279. Min Act: -0.1700

Data Index: 5957957 (Open Web Text)

Max Activating Token Index: 634

Text #2

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.2234. Min Act: -0.1700

Data Index: 4362250 (Open Web Text)

Max Activating Token Index: 624

Text #3

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.2301. Min Act: -0.1700

Data Index: 1650860 (Open Web Text)

Max Activating Token Index: 256

Text #4

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1743. Min Act: -0.1700

Data Index: 7778271 (Open Web Text)

Max Activating Token Index: 697

Text #5

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1795. Min Act: -0.1700

Data Index: 2287946 (Open Web Text)

Max Activating Token Index: 953

Text #6

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1703. Min Act: -0.1700

Data Index: 6467933 (Open Web Text)

Max Activating Token Index: 734

Text #7

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1757. Min Act: -0.1700

Data Index: 7124488 (Open Web Text)

Max Activating Token Index: 118

Text #8

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1776. Min Act: -0.1700

Data Index: 4553138 (Open Web Text)

Max Activating Token Index: 953

Text #9

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1401. Min Act: -0.1700

Data Index: 5706172 (Open Web Text)

Max Activating Token Index: 104

Text #10

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1491. Min Act: -0.1700

Data Index: 3933844 (Open Web Text)

Max Activating Token Index: 353

Text #11

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1301. Min Act: -0.1700

Data Index: 402460 (Open Web Text)

Max Activating Token Index: 577

Text #12

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1279. Min Act: -0.1700

Data Index: 6491321 (Open Web Text)

Max Activating Token Index: 583

Text #13

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1110. Min Act: -0.1700

Data Index: 1341484 (Open Web Text)

Max Activating Token Index: 27

Text #14

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1129. Min Act: -0.1700

Data Index: 4728169 (Open Web Text)

Max Activating Token Index: 547

Text #15

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0950. Min Act: -0.1700

Data Index: 6767256 (Open Web Text)

Max Activating Token Index: 549

Text #16

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.1083. Min Act: -0.1700

Data Index: 8536325 (Open Web Text)

Max Activating Token Index: 998

Text #17

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0636. Min Act: -0.1700

Data Index: 839934 (Open Web Text)

Max Activating Token Index: 533

Text #18

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0635. Min Act: -0.1700

Data Index: 6076552 (Open Web Text)

Max Activating Token Index: 952

Text #19

Max Range: 3.2855. Min Range: -3.2855

Max Act: 3.0916. Min Act: -0.1700

Data Index: 7289610 (Open Web Text)

Max Activating Token Index: 763