Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 878 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.5410. Min Act: -0.1700

Data Index: 1638129 (Python Code)

Max Activating Token Index: 816

Text #1

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.5172. Min Act: -0.1700

Data Index: 1406997 (Python Code)

Max Activating Token Index: 1011

Text #2

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.3800. Min Act: -0.1700

Data Index: 1362825 (Python Code)

Max Activating Token Index: 99

Text #3

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.3062. Min Act: -0.1700

Data Index: 1631490 (Python Code)

Max Activating Token Index: 355

Text #4

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.3160. Min Act: -0.1700

Data Index: 1370743 (Python Code)

Max Activating Token Index: 351

Text #5

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.2956. Min Act: -0.1700

Data Index: 1379465 (Python Code)

Max Activating Token Index: 383

Text #6

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.2912. Min Act: -0.1700

Data Index: 1402098 (Python Code)

Max Activating Token Index: 941

Text #7

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.3234. Min Act: -0.1700

Data Index: 1467911 (Python Code)

Max Activating Token Index: 761

Text #8

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.2494. Min Act: -0.1700

Data Index: 1413199 (Python Code)

Max Activating Token Index: 233

Text #9

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.2287. Min Act: -0.1700

Data Index: 1604143 (Python Code)

Max Activating Token Index: 755

Text #10

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.2152. Min Act: -0.1700

Data Index: 1615173 (Python Code)

Max Activating Token Index: 94

Text #11

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.2086. Min Act: -0.1700

Data Index: 1406138 (Python Code)

Max Activating Token Index: 777

Text #12

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.1842. Min Act: -0.1700

Data Index: 1552248 (Python Code)

Max Activating Token Index: 839

Text #13

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.1935. Min Act: -0.1700

Data Index: 1366021 (Python Code)

Max Activating Token Index: 398

Text #14

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.2254. Min Act: -0.1700

Data Index: 1572501 (Python Code)

Max Activating Token Index: 174

Text #15

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.1912. Min Act: -0.1700

Data Index: 1363937 (Python Code)

Max Activating Token Index: 846

Text #16

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.2015. Min Act: -0.1700

Data Index: 1395055 (Python Code)

Max Activating Token Index: 228

Text #17

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.1758. Min Act: -0.1700

Data Index: 1553489 (Python Code)

Max Activating Token Index: 512

Text #18

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.1989. Min Act: -0.1700

Data Index: 1494446 (Python Code)

Max Activating Token Index: 145

Text #19

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.2131. Min Act: -0.1700

Data Index: 1478317 (Python Code)

Max Activating Token Index: 912